You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2024/09/02 14:54:53 UTC
(datafusion-python) branch asf-staging updated: Publish built docs triggered by 90f5b5b355d79b56ae3607b7b0cdeb09b67e5121
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/datafusion-python.git
The following commit(s) were added to refs/heads/asf-staging by this push:
new 75e0773 Publish built docs triggered by 90f5b5b355d79b56ae3607b7b0cdeb09b67e5121
75e0773 is described below
commit 75e0773bc5c1819f8c96f0092243ec4eb137597c
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Mon Sep 2 14:54:50 2024 +0000
Publish built docs triggered by 90f5b5b355d79b56ae3607b7b0cdeb09b67e5121
---
_sources/autoapi/datafusion/expr/index.rst.txt | 68 +++-
.../autoapi/datafusion/functions/index.rst.txt | 219 ++++++++++-
_sources/autoapi/datafusion/index.rst.txt | 68 +++-
_sources/autoapi/datafusion/udf/index.rst.txt | 2 +-
.../common-operations/aggregations.rst.txt | 2 +
.../user-guide/common-operations/windows.rst.txt | 187 ++++++++--
autoapi/datafusion/expr/index.html | 125 ++++++-
autoapi/datafusion/functions/index.html | 413 +++++++++++++++++++--
autoapi/datafusion/index.html | 125 ++++++-
autoapi/datafusion/udf/index.html | 2 +-
genindex.html | 80 +++-
objects.inv | Bin 4805 -> 4948 bytes
searchindex.js | 2 +-
user-guide/common-operations/aggregations.html | 36 +-
user-guide/common-operations/basic-info.html | 34 +-
user-guide/common-operations/functions.html | 22 +-
user-guide/common-operations/index.html | 7 +-
user-guide/common-operations/joins.html | 6 +-
.../common-operations/select-and-filter.html | 2 +-
user-guide/common-operations/windows.html | 346 +++++++++++++----
20 files changed, 1545 insertions(+), 201 deletions(-)
diff --git a/_sources/autoapi/datafusion/expr/index.rst.txt b/_sources/autoapi/datafusion/expr/index.rst.txt
index a8dadf1..9ee0e94 100644
--- a/_sources/autoapi/datafusion/expr/index.rst.txt
+++ b/_sources/autoapi/datafusion/expr/index.rst.txt
@@ -306,6 +306,32 @@ Module Contents
+ .. py:method:: distinct() -> ExprFuncBuilder
+
+ Only evaluate distinct values for an aggregate function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
+ .. py:method:: filter(filter: Expr) -> ExprFuncBuilder
+
+ Filter an aggregate function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
+ .. py:method:: is_not_null() -> Expr
+
+ Returns ``True`` if this expression is not null.
+
+
+
.. py:method:: is_null() -> Expr
Returns ``True`` if this expression is null.
@@ -322,6 +348,36 @@ Module Contents
+ .. py:method:: null_treatment(null_treatment: datafusion.common.NullTreatment) -> ExprFuncBuilder
+
+ Set the treatment for ``null`` values for a window or aggregate function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
+ .. py:method:: order_by(*exprs: Expr) -> ExprFuncBuilder
+
+ Set the ordering for a window or aggregate function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
+ .. py:method:: partition_by(*partition_by: Expr) -> ExprFuncBuilder
+
+ Set the partitioning for a window function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
.. py:method:: python_value() -> Any
Extracts the Expr value into a PyObject.
@@ -391,6 +447,16 @@ Module Contents
+ .. py:method:: window_frame(window_frame: WindowFrame) -> ExprFuncBuilder
+
+ Set the frame fora window function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
.. py:attribute:: __radd__
@@ -415,7 +481,7 @@ Module Contents
.. py:attribute:: expr
-.. py:class:: WindowFrame(units: str, start_bound: int | None, end_bound: int | None)
+.. py:class:: WindowFrame(units: str, start_bound: Optional[Any], end_bound: Optional[Any])
Defines a window frame for performing window operations.
diff --git a/_sources/autoapi/datafusion/functions/index.rst.txt b/_sources/autoapi/datafusion/functions/index.rst.txt
index 087157b..b6dbabb 100644
--- a/_sources/autoapi/datafusion/functions/index.rst.txt
+++ b/_sources/autoapi/datafusion/functions/index.rst.txt
@@ -93,6 +93,7 @@ Functions
datafusion.functions.covar
datafusion.functions.covar_pop
datafusion.functions.covar_samp
+ datafusion.functions.cume_dist
datafusion.functions.current_date
datafusion.functions.current_time
datafusion.functions.date_bin
@@ -102,6 +103,7 @@ Functions
datafusion.functions.datetrunc
datafusion.functions.decode
datafusion.functions.degrees
+ datafusion.functions.dense_rank
datafusion.functions.digest
datafusion.functions.encode
datafusion.functions.ends_with
@@ -118,8 +120,10 @@ Functions
datafusion.functions.initcap
datafusion.functions.isnan
datafusion.functions.iszero
+ datafusion.functions.lag
datafusion.functions.last_value
datafusion.functions.lcm
+ datafusion.functions.lead
datafusion.functions.left
datafusion.functions.length
datafusion.functions.levenshtein
@@ -167,16 +171,19 @@ Functions
datafusion.functions.named_struct
datafusion.functions.nanvl
datafusion.functions.now
+ datafusion.functions.ntile
datafusion.functions.nullif
datafusion.functions.octet_length
datafusion.functions.order_by
datafusion.functions.overlay
+ datafusion.functions.percent_rank
datafusion.functions.pi
datafusion.functions.pow
datafusion.functions.power
datafusion.functions.radians
datafusion.functions.random
datafusion.functions.range
+ datafusion.functions.rank
datafusion.functions.regexp_like
datafusion.functions.regexp_match
datafusion.functions.regexp_replace
@@ -194,6 +201,7 @@ Functions
datafusion.functions.reverse
datafusion.functions.right
datafusion.functions.round
+ datafusion.functions.row_number
datafusion.functions.rpad
datafusion.functions.rtrim
datafusion.functions.sha224
@@ -688,6 +696,28 @@ Module Contents
Computes the sample covariance.
+.. py:function:: cume_dist(partition_by: Optional[list[datafusion.expr.Expr]] = None, order_by: Optional[list[datafusion.expr.Expr]] = None) -> datafusion.expr.Expr
+
+ Create a cumulative distribution window function.
+
+ This window function is similar to :py:func:`rank` except that the returned values
+ are the ratio of the row number to the total numebr of rows. Here is an example of a
+ dataframe with a window ordered by descending ``points`` and the associated
+ cumulative distribution::
+
+ +--------+-----------+
+ | points | cume_dist |
+ +--------+-----------+
+ | 100 | 0.5 |
+ | 100 | 0.5 |
+ | 50 | 0.75 |
+ | 25 | 1.0 |
+ +--------+-----------+
+
+ :param partition_by: Expressions to partition the window frame on.
+ :param order_by: Set ordering within the window frame.
+
+
.. py:function:: current_date() -> datafusion.expr.Expr
Returns current UTC date as a Date32 value.
@@ -737,6 +767,27 @@ Module Contents
Converts the argument from radians to degrees.
+.. py:function:: dense_rank(partition_by: Optional[list[datafusion.expr.Expr]] = None, order_by: Optional[list[datafusion.expr.Expr]] = None) -> datafusion.expr.Expr
+
+ Create a dense_rank window function.
+
+ This window function is similar to :py:func:`rank` except that the returned values
+ will be consecutive. Here is an example of a dataframe with a window ordered by
+ descending ``points`` and the associated dense rank::
+
+ +--------+------------+
+ | points | dense_rank |
+ +--------+------------+
+ | 100 | 1 |
+ | 100 | 1 |
+ | 50 | 2 |
+ | 25 | 3 |
+ +--------+------------+
+
+ :param partition_by: Expressions to partition the window frame on.
+ :param order_by: Set ordering within the window frame.
+
+
.. py:function:: digest(value: datafusion.expr.Expr, method: datafusion.expr.Expr) -> datafusion.expr.Expr
Computes the binary hash of an expression using the specified algorithm.
@@ -775,7 +826,7 @@ Module Contents
The string list is a string composed of substrings separated by ``,`` characters.
-.. py:function:: first_value(arg: datafusion.expr.Expr, distinct: bool = False, filter: bool = None, order_by: datafusion.expr.Expr | None = None, null_treatment: datafusion._internal.common.NullTreatment | None = None) -> datafusion.expr.Expr
+.. py:function:: first_value(arg: datafusion.expr.Expr, distinct: bool = False, filter: Optional[bool] = None, order_by: Optional[list[datafusion.expr.Expr]] = None, null_treatment: Optional[datafusion._internal.common.NullTreatment] = None) -> datafusion.expr.Expr
Returns the first value in a group of values.
@@ -830,16 +881,78 @@ Module Contents
Returns true if a given number is +0.0 or -0.0 otherwise returns false.
-.. py:function:: last_value(arg: datafusion.expr.Expr, distinct: bool = False, filter: bool = None, order_by: datafusion.expr.Expr | None = None, null_treatment: datafusion._internal.common.NullTreatment | None = None) -> datafusion.expr.Expr
+.. py:function:: lag(arg: datafusion.expr.Expr, shift_offset: int = 1, default_value: Optional[Any] = None, partition_by: Optional[list[datafusion.expr.Expr]] = None, order_by: Optional[list[datafusion.expr.Expr]] = None) -> datafusion.expr.Expr
+
+ Create a lag window function.
+
+ Lag operation will return the argument that is in the previous shift_offset-th row
+ in the partition. For example ``lag(col("b"), shift_offset=3, default_value=5)``
+ will return the 3rd previous value in column ``b``. At the beginnig of the
+ partition, where no values can be returned it will return the default value of 5.
+
+ Here is an example of both the ``lag`` and :py:func:`datafusion.functions.lead`
+ functions on a simple DataFrame::
+
+ +--------+------+-----+
+ | points | lead | lag |
+ +--------+------+-----+
+ | 100 | 100 | |
+ | 100 | 50 | 100 |
+ | 50 | 25 | 100 |
+ | 25 | | 50 |
+ +--------+------+-----+
+
+ :param arg: Value to return
+ :param shift_offset: Number of rows before the current row.
+ :param default_value: Value to return if shift_offet row does not exist.
+ :param partition_by: Expressions to partition the window frame on.
+ :param order_by: Set ordering within the window frame.
+
+
+.. py:function:: last_value(arg: datafusion.expr.Expr, distinct: bool = False, filter: Optional[bool] = None, order_by: Optional[list[datafusion.expr.Expr]] = None, null_treatment: Optional[datafusion._internal.common.NullTreatment] = None) -> datafusion.expr.Expr
Returns the last value in a group of values.
+ To set parameters on this expression, use ``.order_by()``, ``.distinct()``,
+ ``.filter()``, or ``.null_treatment()``.
+
.. py:function:: lcm(x: datafusion.expr.Expr, y: datafusion.expr.Expr) -> datafusion.expr.Expr
Returns the least common multiple.
+.. py:function:: lead(arg: datafusion.expr.Expr, shift_offset: int = 1, default_value: Optional[Any] = None, partition_by: Optional[list[datafusion.expr.Expr]] = None, order_by: Optional[list[datafusion.expr.Expr]] = None) -> datafusion.expr.Expr
+
+ Create a lead window function.
+
+ Lead operation will return the argument that is in the next shift_offset-th row in
+ the partition. For example ``lead(col("b"), shift_offset=3, default_value=5)`` will
+ return the 3rd following value in column ``b``. At the end of the partition, where
+ no futher values can be returned it will return the default value of 5.
+
+ Here is an example of both the ``lead`` and :py:func:`datafusion.functions.lag`
+ functions on a simple DataFrame::
+
+ +--------+------+-----+
+ | points | lead | lag |
+ +--------+------+-----+
+ | 100 | 100 | |
+ | 100 | 50 | 100 |
+ | 50 | 25 | 100 |
+ | 25 | | 50 |
+ +--------+------+-----+
+
+ To set window function parameters use the window builder approach described in the
+ ref:`_window_functions` online documentation.
+
+ :param arg: Value to return
+ :param shift_offset: Number of rows following the current row.
+ :param default_value: Value to return if shift_offet row does not exist.
+ :param partition_by: Expressions to partition the window frame on.
+ :param order_by: Set ordering within the window frame.
+
+
.. py:function:: left(string: datafusion.expr.Expr, n: datafusion.expr.Expr) -> datafusion.expr.Expr
Returns the first ``n`` characters in the ``string``.
@@ -1124,7 +1237,7 @@ Module Contents
Returns the minimum value of the argument.
-.. py:function:: named_struct(name_pairs: list[str, datafusion.expr.Expr]) -> datafusion.expr.Expr
+.. py:function:: named_struct(name_pairs: list[tuple[str, datafusion.expr.Expr]]) -> datafusion.expr.Expr
Returns a struct with the given names and arguments pairs.
@@ -1141,6 +1254,31 @@ Module Contents
This will use the same value for all instances of now() in same statement.
+.. py:function:: ntile(groups: int, partition_by: Optional[list[datafusion.expr.Expr]] = None, order_by: Optional[list[datafusion.expr.Expr]] = None) -> datafusion.expr.Expr
+
+ Create a n-tile window function.
+
+ This window function orders the window frame into a give number of groups based on
+ the ordering criteria. It then returns which group the current row is assigned to.
+ Here is an example of a dataframe with a window ordered by descending ``points``
+ and the associated n-tile function::
+
+ +--------+-------+
+ | points | ntile |
+ +--------+-------+
+ | 120 | 1 |
+ | 100 | 1 |
+ | 80 | 2 |
+ | 60 | 2 |
+ | 40 | 3 |
+ | 20 | 3 |
+ +--------+-------+
+
+ :param groups: Number of groups for the n-tile to be divided into.
+ :param partition_by: Expressions to partition the window frame on.
+ :param order_by: Set ordering within the window frame.
+
+
.. py:function:: nullif(expr1: datafusion.expr.Expr, expr2: datafusion.expr.Expr) -> datafusion.expr.Expr
Returns NULL if expr1 equals expr2; otherwise it returns expr1.
@@ -1166,6 +1304,28 @@ Module Contents
extends for ``length`` characters with new substring.
+.. py:function:: percent_rank(partition_by: Optional[list[datafusion.expr.Expr]] = None, order_by: Optional[list[datafusion.expr.Expr]] = None) -> datafusion.expr.Expr
+
+ Create a percent_rank window function.
+
+ This window function is similar to :py:func:`rank` except that the returned values
+ are the percentage from 0.0 to 1.0 from first to last. Here is an example of a
+ dataframe with a window ordered by descending ``points`` and the associated percent
+ rank::
+
+ +--------+--------------+
+ | points | percent_rank |
+ +--------+--------------+
+ | 100 | 0.0 |
+ | 100 | 0.0 |
+ | 50 | 0.666667 |
+ | 25 | 1.0 |
+ +--------+--------------+
+
+ :param partition_by: Expressions to partition the window frame on.
+ :param order_by: Set ordering within the window frame.
+
+
.. py:function:: pi() -> datafusion.expr.Expr
Returns an approximate value of π.
@@ -1198,6 +1358,32 @@ Module Contents
Create a list of values in the range between start and stop.
+.. py:function:: rank(partition_by: Optional[list[datafusion.expr.Expr]] = None, order_by: Optional[list[datafusion.expr.Expr]] = None) -> datafusion.expr.Expr
+
+ Create a rank window function.
+
+ Returns the rank based upon the window order. Consecutive equal values will receive
+ the same rank, but the next different value will not be consecutive but rather the
+ number of rows that preceed it plus one. This is similar to Olympic medals. If two
+ people tie for gold, the next place is bronze. There would be no silver medal. Here
+ is an example of a dataframe with a window ordered by descending ``points`` and the
+ associated rank.
+
+ You should set ``order_by`` to produce meaningful results::
+
+ +--------+------+
+ | points | rank |
+ +--------+------+
+ | 100 | 1 |
+ | 100 | 1 |
+ | 50 | 3 |
+ | 25 | 4 |
+ +--------+------+
+
+ :param partition_by: Expressions to partition the window frame on.
+ :param order_by: Set ordering within the window frame.
+
+
.. py:function:: regexp_like(string: datafusion.expr.Expr, regex: datafusion.expr.Expr, flags: datafusion.expr.Expr | None = None) -> datafusion.expr.Expr
Find if any regular expression (regex) matches exist.
@@ -1303,6 +1489,27 @@ Module Contents
``round(lit(125.2345), lit(-2))`` would yield a value of ``100.0``.
+.. py:function:: row_number(partition_by: Optional[list[datafusion.expr.Expr]] = None, order_by: Optional[list[datafusion.expr.Expr]] = None) -> datafusion.expr.Expr
+
+ Create a row number window function.
+
+ Returns the row number of the window function.
+
+ Here is an example of the ``row_number`` on a simple DataFrame::
+
+ +--------+------------+
+ | points | row number |
+ +--------+------------+
+ | 100 | 1 |
+ | 100 | 2 |
+ | 50 | 3 |
+ | 25 | 4 |
+ +--------+------------+
+
+ :param partition_by: Expressions to partition the window frame on.
+ :param order_by: Set ordering within the window frame.
+
+
.. py:function:: rpad(string: datafusion.expr.Expr, count: datafusion.expr.Expr, characters: datafusion.expr.Expr | None = None) -> datafusion.expr.Expr
Add right padding to a string.
@@ -1524,4 +1731,10 @@ Module Contents
Creates a new Window function expression.
+ This interface will soon be deprecated. Instead of using this interface,
+ users should call the window functions directly. For example, to perform a
+ lag use::
+
+ df.select(functions.lag(col("a")).partition_by(col("b")).build())
+
diff --git a/_sources/autoapi/datafusion/index.rst.txt b/_sources/autoapi/datafusion/index.rst.txt
index a804238..38426ba 100644
--- a/_sources/autoapi/datafusion/index.rst.txt
+++ b/_sources/autoapi/datafusion/index.rst.txt
@@ -380,6 +380,32 @@ Package Contents
+ .. py:method:: distinct() -> ExprFuncBuilder
+
+ Only evaluate distinct values for an aggregate function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
+ .. py:method:: filter(filter: Expr) -> ExprFuncBuilder
+
+ Filter an aggregate function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
+ .. py:method:: is_not_null() -> Expr
+
+ Returns ``True`` if this expression is not null.
+
+
+
.. py:method:: is_null() -> Expr
Returns ``True`` if this expression is null.
@@ -396,6 +422,36 @@ Package Contents
+ .. py:method:: null_treatment(null_treatment: datafusion.common.NullTreatment) -> ExprFuncBuilder
+
+ Set the treatment for ``null`` values for a window or aggregate function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
+ .. py:method:: order_by(*exprs: Expr) -> ExprFuncBuilder
+
+ Set the ordering for a window or aggregate function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
+ .. py:method:: partition_by(*partition_by: Expr) -> ExprFuncBuilder
+
+ Set the partitioning for a window function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
.. py:method:: python_value() -> Any
Extracts the Expr value into a PyObject.
@@ -465,6 +521,16 @@ Package Contents
+ .. py:method:: window_frame(window_frame: WindowFrame) -> ExprFuncBuilder
+
+ Set the frame fora window function.
+
+ This function will create an :py:class:`ExprFuncBuilder` that can be used to
+ set parameters for either window or aggregate functions. If used on any other
+ type of expression, an error will be generated when ``build()`` is called.
+
+
+
.. py:attribute:: __radd__
@@ -927,7 +993,7 @@ Package Contents
.. py:attribute:: table
-.. py:class:: WindowFrame(units: str, start_bound: int | None, end_bound: int | None)
+.. py:class:: WindowFrame(units: str, start_bound: Optional[Any], end_bound: Optional[Any])
Defines a window frame for performing window operations.
diff --git a/_sources/autoapi/datafusion/udf/index.rst.txt b/_sources/autoapi/datafusion/udf/index.rst.txt
index 17310f2..88ccc9c 100644
--- a/_sources/autoapi/datafusion/udf/index.rst.txt
+++ b/_sources/autoapi/datafusion/udf/index.rst.txt
@@ -159,7 +159,7 @@ Module Contents
.. py:attribute:: _udf
-.. py:class:: Volatility
+.. py:class:: Volatility(*args, **kwds)
Bases: :py:obj:`enum.Enum`
diff --git a/_sources/user-guide/common-operations/aggregations.rst.txt b/_sources/user-guide/common-operations/aggregations.rst.txt
index b920212..7ad4022 100644
--- a/_sources/user-guide/common-operations/aggregations.rst.txt
+++ b/_sources/user-guide/common-operations/aggregations.rst.txt
@@ -15,6 +15,8 @@
.. specific language governing permissions and limitations
.. under the License.
+.. _aggregation:
+
Aggregation
============
diff --git a/_sources/user-guide/common-operations/windows.rst.txt b/_sources/user-guide/common-operations/windows.rst.txt
index 5ef3c98..6091768 100644
--- a/_sources/user-guide/common-operations/windows.rst.txt
+++ b/_sources/user-guide/common-operations/windows.rst.txt
@@ -15,13 +15,16 @@
.. specific language governing permissions and limitations
.. under the License.
+.. _window_functions:
+
Window Functions
================
-In this section you will learn about window functions. A window function utilizes values from one or multiple rows to
-produce a result for each individual row, unlike an aggregate function that provides a single value for multiple rows.
+In this section you will learn about window functions. A window function utilizes values from one or
+multiple rows to produce a result for each individual row, unlike an aggregate function that
+provides a single value for multiple rows.
-The functionality of window functions in DataFusion is supported by the dedicated :py:func:`~datafusion.functions.window` function.
+The window functions are availble in the :py:mod:`~datafusion.functions` module.
We'll use the pokemon dataset (from Ritchie Vink) in the following examples.
@@ -40,20 +43,25 @@ We'll use the pokemon dataset (from Ritchie Vink) in the following examples.
ctx = SessionContext()
df = ctx.read_csv("pokemon.csv")
-Here is an example that shows how to compare each pokemons’s attack power with the average attack power in its ``"Type 1"``
+Here is an example that shows how you can compare each pokemon's speed to the speed of the
+previous row in the DataFrame.
.. ipython:: python
df.select(
col('"Name"'),
- col('"Attack"'),
- f.alias(
- f.window("avg", [col('"Attack"')], partition_by=[col('"Type 1"')]),
- "Average Attack",
- )
+ col('"Speed"'),
+ f.lag(col('"Speed"')).alias("Previous Speed")
)
-You can also control the order in which rows are processed by window functions by providing
+Setting Parameters
+------------------
+
+
+Ordering
+^^^^^^^^
+
+You can control the order in which rows are processed by window functions by providing
a list of ``order_by`` functions for the ``order_by`` parameter.
.. ipython:: python
@@ -61,33 +69,150 @@ a list of ``order_by`` functions for the ``order_by`` parameter.
df.select(
col('"Name"'),
col('"Attack"'),
- f.alias(
- f.window(
- "rank",
- [],
- partition_by=[col('"Type 1"')],
- order_by=[f.order_by(col('"Attack"'))],
- ),
- "rank",
- ),
+ col('"Type 1"'),
+ f.rank(
+ partition_by=[col('"Type 1"')],
+ order_by=[col('"Attack"').sort(ascending=True)],
+ ).alias("rank"),
+ ).sort(col('"Type 1"'), col('"Attack"'))
+
+Partitions
+^^^^^^^^^^
+
+A window function can take a list of ``partition_by`` columns similar to an
+:ref:`Aggregation Function<aggregation>`. This will cause the window values to be evaluated
+independently for each of the partitions. In the example above, we found the rank of each
+Pokemon per ``Type 1`` partitions. We can see the first couple of each partition if we do
+the following:
+
+.. ipython:: python
+
+ df.select(
+ col('"Name"'),
+ col('"Attack"'),
+ col('"Type 1"'),
+ f.rank(
+ partition_by=[col('"Type 1"')],
+ order_by=[col('"Attack"').sort(ascending=True)],
+ ).alias("rank"),
+ ).filter(col("rank") < lit(3)).sort(col('"Type 1"'), col("rank"))
+
+Window Frame
+^^^^^^^^^^^^
+
+When using aggregate functions, the Window Frame of defines the rows over which it operates.
+If you do not specify a Window Frame, the frame will be set depending on the following
+criteria.
+
+* If an ``order_by`` clause is set, the default window frame is defined as the rows between
+ unbounded preceeding and the current row.
+* If an ``order_by`` is not set, the default frame is defined as the rows betwene unbounded
+ and unbounded following (the entire partition).
+
+Window Frames are defined by three parameters: unit type, starting bound, and ending bound.
+
+The unit types available are:
+
+* Rows: The starting and ending boundaries are defined by the number of rows relative to the
+ current row.
+* Range: When using Range, the ``order_by`` clause must have exactly one term. The boundaries
+ are defined bow how close the rows are to the value of the expression in the ``order_by``
+ parameter.
+* Groups: A "group" is the set of all rows that have equivalent values for all terms in the
+ ``order_by`` clause.
+
+In this example we perform a "rolling average" of the speed of the current Pokemon and the
+two preceeding rows.
+
+.. ipython:: python
+
+ from datafusion.expr import WindowFrame
+
+ df.select(
+ col('"Name"'),
+ col('"Speed"'),
+ f.window("avg",
+ [col('"Speed"')],
+ order_by=[col('"Speed"')],
+ window_frame=WindowFrame("rows", 2, 0)
+ ).alias("Previous Speed")
+ )
+
+Null Treatment
+^^^^^^^^^^^^^^
+
+When using aggregate functions as window functions, it is often useful to specify how null values
+should be treated. In order to do this you need to use the builder function. In future releases
+we expect this to be simplified in the interface.
+
+One common usage for handling nulls is the case where you want to find the last value up to the
+current row. In the following example we demonstrate how setting the null treatment to ignore
+nulls will fill in with the value of the most recent non-null row. To do this, we also will set
+the window frame so that we only process up to the current row.
+
+In this example, we filter down to one specific type of Pokemon that does have some entries in
+it's ``Type 2`` column that are null.
+
+.. ipython:: python
+
+ from datafusion.common import NullTreatment
+
+ df.filter(col('"Type 1"') == lit("Bug")).select(
+ '"Name"',
+ '"Type 2"',
+ f.window("last_value", [col('"Type 2"')])
+ .window_frame(WindowFrame("rows", None, 0))
+ .order_by(col('"Speed"'))
+ .null_treatment(NullTreatment.IGNORE_NULLS)
+ .build()
+ .alias("last_wo_null"),
+ f.window("last_value", [col('"Type 2"')])
+ .window_frame(WindowFrame("rows", None, 0))
+ .order_by(col('"Speed"'))
+ .null_treatment(NullTreatment.RESPECT_NULLS)
+ .build()
+ .alias("last_with_null")
+ )
+
+Aggregate Functions
+-------------------
+
+You can use any :ref:`Aggregation Function<aggregation>` as a window function. Currently
+aggregate functions must use the deprecated
+:py:func:`datafusion.functions.window` API but this should be resolved in
+DataFusion 42.0 (`Issue Link <https://github.com/apache/datafusion-python/issues/833>`_). Here
+is an example that shows how to compare each pokemons’s attack power with the average attack
+power in its ``"Type 1"`` using the :py:func:`datafusion.functions.avg` function.
+
+.. ipython:: python
+ :okwarning:
+
+ df.select(
+ col('"Name"'),
+ col('"Attack"'),
+ col('"Type 1"'),
+ f.window("avg", [col('"Attack"')])
+ .partition_by(col('"Type 1"'))
+ .build()
+ .alias("Average Attack"),
)
+Available Functions
+-------------------
+
The possible window functions are:
1. Rank Functions
- - rank
- - dense_rank
- - row_number
- - ntile
+ - :py:func:`datafusion.functions.rank`
+ - :py:func:`datafusion.functions.dense_rank`
+ - :py:func:`datafusion.functions.ntile`
+ - :py:func:`datafusion.functions.row_number`
2. Analytical Functions
- - cume_dist
- - percent_rank
- - lag
- - lead
- - first_value
- - last_value
- - nth_value
+ - :py:func:`datafusion.functions.cume_dist`
+ - :py:func:`datafusion.functions.percent_rank`
+ - :py:func:`datafusion.functions.lag`
+ - :py:func:`datafusion.functions.lead`
3. Aggregate Functions
- - All aggregate functions can be used as window functions.
+ - All :ref:`Aggregation Functions<aggregation>` can be used as window functions.
diff --git a/autoapi/datafusion/expr/index.html b/autoapi/datafusion/expr/index.html
index c0e1d8e..9955402 100644
--- a/autoapi/datafusion/expr/index.html
+++ b/autoapi/datafusion/expr/index.html
@@ -623,6 +623,33 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.expr.Expr.distinct">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.distinct()
+ </span>
+ </code>
+ </a>
+ </li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.expr.Expr.filter">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.filter()
+ </span>
+ </code>
+ </a>
+ </li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.expr.Expr.is_not_null">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.is_not_null()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h4 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.expr.Expr.is_null">
<code class="docutils literal notranslate">
@@ -641,6 +668,33 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.expr.Expr.null_treatment">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.null_treatment()
+ </span>
+ </code>
+ </a>
+ </li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.expr.Expr.order_by">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.order_by()
+ </span>
+ </code>
+ </a>
+ </li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.expr.Expr.partition_by">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.partition_by()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h4 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.expr.Expr.python_value">
<code class="docutils literal notranslate">
@@ -713,6 +767,15 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.expr.Expr.window_frame">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.window_frame()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h4 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.expr.Expr.__radd__">
<code class="docutils literal notranslate">
@@ -1827,6 +1890,30 @@
<p>This name will not include any CAST expressions.</p>
</dd></dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.expr.Expr.distinct">
+<span class="sig-name descname"><span class="pre">distinct</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">ExprFuncBuilder</span></span></span><a class="headerlink" href="#datafusion.expr.Expr.distinct" title="Link to this definition">¶</a></dt>
+<dd><p>Only evaluate distinct values for an aggregate function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.expr.Expr.filter">
+<span class="sig-name descname"><span class="pre">filter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">filter</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">Expr</span></a></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon"> [...]
+<dd><p>Filter an aggregate function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.expr.Expr.is_not_null">
+<span class="sig-name descname"><span class="pre">is_not_null</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">Expr</span></a></span></span><a class="headerlink" href="#datafusion.expr.Expr.is_not_null" title="Link to this definition">¶</a></dt>
+<dd><p>Returns <code class="docutils literal notranslate"><span class="pre">True</span></code> if this expression is not null.</p>
+</dd></dl>
+
<dl class="py method">
<dt class="sig sig-object py" id="datafusion.expr.Expr.is_null">
<span class="sig-name descname"><span class="pre">is_null</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">Expr</span></a></span></span><a class="headerlink" href="#datafusion.expr.Expr.is_null" title="Link to this definition">¶</a></dt>
@@ -1840,6 +1927,33 @@
<p><code class="docutils literal notranslate"><span class="pre">value</span></code> must be a valid PyArrow scalar value or easily castable to one.</p>
</dd></dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.expr.Expr.null_treatment">
+<span class="sig-name descname"><span class="pre">null_treatment</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">null_treatment</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">datafusion.common.NullTreatment</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><sp [...]
+<dd><p>Set the treatment for <code class="docutils literal notranslate"><span class="pre">null</span></code> values for a window or aggregate function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.expr.Expr.order_by">
+<span class="sig-name descname"><span class="pre">order_by</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">exprs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">Expr</span></a></span></em><span class="sig-paren">)</span> <span cla [...]
+<dd><p>Set the ordering for a window or aggregate function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.expr.Expr.partition_by">
+<span class="sig-name descname"><span class="pre">partition_by</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">Expr</span></a></span></em><span class="sig-paren">)</span [...]
+<dd><p>Set the partitioning for a window function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
<dl class="py method">
<dt class="sig sig-object py" id="datafusion.expr.Expr.python_value">
<span class="sig-name descname"><span class="pre">python_value</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">Any</span></span></span><a class="headerlink" href="#datafusion.expr.Expr.python_value" title="Link to this definition">¶</a></dt>
@@ -1916,6 +2030,15 @@ SqlType Enum which this expression represents.</p>
<p>Ex: <code class="docutils literal notranslate"><span class="pre">IsNotNull</span></code>, <code class="docutils literal notranslate"><span class="pre">Literal</span></code>, <code class="docutils literal notranslate"><span class="pre">BinaryExpr</span></code>, etc</p>
</dd></dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.expr.Expr.window_frame">
+<span class="sig-name descname"><span class="pre">window_frame</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">window_frame</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.expr.WindowFrame" title="datafusion.expr.WindowFrame"><span class="pre">WindowFrame</span></a></span></em><span class="sig-paren">)</span> <span class="sig-return">< [...]
+<dd><p>Set the frame fora window function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
<dl class="py attribute">
<dt class="sig sig-object py" id="datafusion.expr.Expr.__radd__">
<span class="sig-name descname"><span class="pre">__radd__</span></span><a class="headerlink" href="#datafusion.expr.Expr.__radd__" title="Link to this definition">¶</a></dt>
@@ -1960,7 +2083,7 @@ SqlType Enum which this expression represents.</p>
<dl class="py class">
<dt class="sig sig-object py" id="datafusion.expr.WindowFrame">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">WindowFrame</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">units</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class=" [...]
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">WindowFrame</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">units</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class=" [...]
<dd><p>Defines a window frame for performing window operations.</p>
<p>Construct a window frame using the given parameters.</p>
<dl class="field-list simple">
diff --git a/autoapi/datafusion/functions/index.html b/autoapi/datafusion/functions/index.html
index 5e53dcc..eccebcc 100644
--- a/autoapi/datafusion/functions/index.html
+++ b/autoapi/datafusion/functions/index.html
@@ -1066,6 +1066,15 @@
</code>
</a>
</li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.functions.cume_dist">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ cume_dist()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.functions.current_date">
<code class="docutils literal notranslate">
@@ -1147,6 +1156,15 @@
</code>
</a>
</li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.functions.dense_rank">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ dense_rank()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.functions.digest">
<code class="docutils literal notranslate">
@@ -1291,6 +1309,15 @@
</code>
</a>
</li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.functions.lag">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ lag()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.functions.last_value">
<code class="docutils literal notranslate">
@@ -1309,6 +1336,15 @@
</code>
</a>
</li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.functions.lead">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ lead()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.functions.left">
<code class="docutils literal notranslate">
@@ -1732,6 +1768,15 @@
</code>
</a>
</li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.functions.ntile">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ ntile()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.functions.nullif">
<code class="docutils literal notranslate">
@@ -1768,6 +1813,15 @@
</code>
</a>
</li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.functions.percent_rank">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ percent_rank()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.functions.pi">
<code class="docutils literal notranslate">
@@ -1822,6 +1876,15 @@
</code>
</a>
</li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.functions.rank">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ rank()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.functions.regexp_like">
<code class="docutils literal notranslate">
@@ -1975,6 +2038,15 @@
</code>
</a>
</li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.functions.row_number">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ row_number()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.functions.rpad">
<code class="docutils literal notranslate">
@@ -2593,33 +2665,39 @@
<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.covar_samp" title="datafusion.functions.covar_samp"><code class="xref py py-obj docutils literal notranslate"><span class="pre">covar_samp</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the sample covariance.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.current_date" title="datafusion.functions.current_date"><code class="xref py py-obj docutils literal notranslate"><span class="pre">current_date</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.cume_dist" title="datafusion.functions.cume_dist"><code class="xref py py-obj docutils literal notranslate"><span class="pre">cume_dist</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<td><p>Create a cumulative distribution window function.</p></td>
+</tr>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.current_date" title="datafusion.functions.current_date"><code class="xref py py-obj docutils literal notranslate"><span class="pre">current_date</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns current UTC date as a Date32 value.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.current_time" title="datafusion.functions.current_time"><code class="xref py py-obj docutils literal notranslate"><span class="pre">current_time</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.current_time" title="datafusion.functions.current_time"><code class="xref py py-obj docutils literal notranslate"><span class="pre">current_time</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns current UTC time as a Time64 value.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.date_bin" title="datafusion.functions.date_bin"><code class="xref py py-obj docutils literal notranslate"><span class="pre">date_bin</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.date_bin" title="datafusion.functions.date_bin"><code class="xref py py-obj docutils literal notranslate"><span class="pre">date_bin</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Coerces an arbitrary timestamp to the start of the nearest specified interval.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.date_part" title="datafusion.functions.date_part"><code class="xref py py-obj docutils literal notranslate"><span class="pre">date_part</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.date_part" title="datafusion.functions.date_part"><code class="xref py py-obj docutils literal notranslate"><span class="pre">date_part</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Extracts a subfield from the date.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.date_trunc" title="datafusion.functions.date_trunc"><code class="xref py py-obj docutils literal notranslate"><span class="pre">date_trunc</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.date_trunc" title="datafusion.functions.date_trunc"><code class="xref py py-obj docutils literal notranslate"><span class="pre">date_trunc</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Truncates the date to a specified level of precision.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.datepart" title="datafusion.functions.datepart"><code class="xref py py-obj docutils literal notranslate"><span class="pre">datepart</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.datepart" title="datafusion.functions.datepart"><code class="xref py py-obj docutils literal notranslate"><span class="pre">datepart</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Return a specified part of a date.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.datetrunc" title="datafusion.functions.datetrunc"><code class="xref py py-obj docutils literal notranslate"><span class="pre">datetrunc</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.datetrunc" title="datafusion.functions.datetrunc"><code class="xref py py-obj docutils literal notranslate"><span class="pre">datetrunc</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Truncates the date to a specified level of precision.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.decode" title="datafusion.functions.decode"><code class="xref py py-obj docutils literal notranslate"><span class="pre">decode</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.decode" title="datafusion.functions.decode"><code class="xref py py-obj docutils literal notranslate"><span class="pre">decode</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Decode the <code class="docutils literal notranslate"><span class="pre">input</span></code>, using the <code class="docutils literal notranslate"><span class="pre">encoding</span></code>. encoding can be base64 or hex.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.degrees" title="datafusion.functions.degrees"><code class="xref py py-obj docutils literal notranslate"><span class="pre">degrees</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.degrees" title="datafusion.functions.degrees"><code class="xref py py-obj docutils literal notranslate"><span class="pre">degrees</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Converts the argument from radians to degrees.</p></td>
</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.dense_rank" title="datafusion.functions.dense_rank"><code class="xref py py-obj docutils literal notranslate"><span class="pre">dense_rank</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<td><p>Create a dense_rank window function.</p></td>
+</tr>
<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.digest" title="datafusion.functions.digest"><code class="xref py py-obj docutils literal notranslate"><span class="pre">digest</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the binary hash of an expression using the specified algorithm.</p></td>
</tr>
@@ -2668,12 +2746,18 @@
<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.iszero" title="datafusion.functions.iszero"><code class="xref py py-obj docutils literal notranslate"><span class="pre">iszero</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns true if a given number is +0.0 or -0.0 otherwise returns false.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.last_value" title="datafusion.functions.last_value"><code class="xref py py-obj docutils literal notranslate"><span class="pre">last_value</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.lag" title="datafusion.functions.lag"><code class="xref py py-obj docutils literal notranslate"><span class="pre">lag</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<td><p>Create a lag window function.</p></td>
+</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.last_value" title="datafusion.functions.last_value"><code class="xref py py-obj docutils literal notranslate"><span class="pre">last_value</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns the last value in a group of values.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.lcm" title="datafusion.functions.lcm"><code class="xref py py-obj docutils literal notranslate"><span class="pre">lcm</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.lcm" title="datafusion.functions.lcm"><code class="xref py py-obj docutils literal notranslate"><span class="pre">lcm</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns the least common multiple.</p></td>
</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.lead" title="datafusion.functions.lead"><code class="xref py py-obj docutils literal notranslate"><span class="pre">lead</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<td><p>Create a lead window function.</p></td>
+</tr>
<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.left" title="datafusion.functions.left"><code class="xref py py-obj docutils literal notranslate"><span class="pre">left</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns the first <code class="docutils literal notranslate"><span class="pre">n</span></code> characters in the <code class="docutils literal notranslate"><span class="pre">string</span></code>.</p></td>
</tr>
@@ -2815,18 +2899,24 @@
<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.now" title="datafusion.functions.now"><code class="xref py py-obj docutils literal notranslate"><span class="pre">now</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns the current timestamp in nanoseconds.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.nullif" title="datafusion.functions.nullif"><code class="xref py py-obj docutils literal notranslate"><span class="pre">nullif</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.ntile" title="datafusion.functions.ntile"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ntile</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<td><p>Create a n-tile window function.</p></td>
+</tr>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.nullif" title="datafusion.functions.nullif"><code class="xref py py-obj docutils literal notranslate"><span class="pre">nullif</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns NULL if expr1 equals expr2; otherwise it returns expr1.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.octet_length" title="datafusion.functions.octet_length"><code class="xref py py-obj docutils literal notranslate"><span class="pre">octet_length</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.octet_length" title="datafusion.functions.octet_length"><code class="xref py py-obj docutils literal notranslate"><span class="pre">octet_length</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns the number of bytes of a string.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.order_by" title="datafusion.functions.order_by"><code class="xref py py-obj docutils literal notranslate"><span class="pre">order_by</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.order_by" title="datafusion.functions.order_by"><code class="xref py py-obj docutils literal notranslate"><span class="pre">order_by</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Creates a new sort expression.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.overlay" title="datafusion.functions.overlay"><code class="xref py py-obj docutils literal notranslate"><span class="pre">overlay</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.overlay" title="datafusion.functions.overlay"><code class="xref py py-obj docutils literal notranslate"><span class="pre">overlay</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Replace a substring with a new substring.</p></td>
</tr>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.percent_rank" title="datafusion.functions.percent_rank"><code class="xref py py-obj docutils literal notranslate"><span class="pre">percent_rank</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<td><p>Create a percent_rank window function.</p></td>
+</tr>
<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.pi" title="datafusion.functions.pi"><code class="xref py py-obj docutils literal notranslate"><span class="pre">pi</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns an approximate value of π.</p></td>
</tr>
@@ -2845,57 +2935,63 @@
<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.range" title="datafusion.functions.range"><code class="xref py py-obj docutils literal notranslate"><span class="pre">range</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Create a list of values in the range between start and stop.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regexp_like" title="datafusion.functions.regexp_like"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regexp_like</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.rank" title="datafusion.functions.rank"><code class="xref py py-obj docutils literal notranslate"><span class="pre">rank</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<td><p>Create a rank window function.</p></td>
+</tr>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regexp_like" title="datafusion.functions.regexp_like"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regexp_like</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Find if any regular expression (regex) matches exist.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regexp_match" title="datafusion.functions.regexp_match"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regexp_match</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regexp_match" title="datafusion.functions.regexp_match"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regexp_match</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Perform regular expression (regex) matching.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regexp_replace" title="datafusion.functions.regexp_replace"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regexp_replace</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regexp_replace" title="datafusion.functions.regexp_replace"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regexp_replace</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Replaces substring(s) matching a PCRE-like regular expression.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_avgx" title="datafusion.functions.regr_avgx"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_avgx</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_avgx" title="datafusion.functions.regr_avgx"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_avgx</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the average of the independent variable <code class="docutils literal notranslate"><span class="pre">x</span></code>.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_avgy" title="datafusion.functions.regr_avgy"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_avgy</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_avgy" title="datafusion.functions.regr_avgy"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_avgy</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the average of the dependent variable <code class="docutils literal notranslate"><span class="pre">y</span></code>.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_count" title="datafusion.functions.regr_count"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_count</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_count" title="datafusion.functions.regr_count"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_count</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Counts the number of rows in which both expressions are not null.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_intercept" title="datafusion.functions.regr_intercept"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_intercept</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_intercept" title="datafusion.functions.regr_intercept"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_intercept</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the intercept from the linear regression.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_r2" title="datafusion.functions.regr_r2"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_r2</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_r2" title="datafusion.functions.regr_r2"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_r2</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the R-squared value from linear regression.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_slope" title="datafusion.functions.regr_slope"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_slope</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_slope" title="datafusion.functions.regr_slope"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_slope</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the slope from linear regression.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_sxx" title="datafusion.functions.regr_sxx"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_sxx</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_sxx" title="datafusion.functions.regr_sxx"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_sxx</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the sum of squares of the independent variable <code class="docutils literal notranslate"><span class="pre">x</span></code>.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_sxy" title="datafusion.functions.regr_sxy"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_sxy</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_sxy" title="datafusion.functions.regr_sxy"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_sxy</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the sum of products of pairs of numbers.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.regr_syy" title="datafusion.functions.regr_syy"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_syy</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.regr_syy" title="datafusion.functions.regr_syy"><code class="xref py py-obj docutils literal notranslate"><span class="pre">regr_syy</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Computes the sum of squares of the dependent variable <code class="docutils literal notranslate"><span class="pre">y</span></code>.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.repeat" title="datafusion.functions.repeat"><code class="xref py py-obj docutils literal notranslate"><span class="pre">repeat</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.repeat" title="datafusion.functions.repeat"><code class="xref py py-obj docutils literal notranslate"><span class="pre">repeat</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Repeats the <code class="docutils literal notranslate"><span class="pre">string</span></code> to <code class="docutils literal notranslate"><span class="pre">n</span></code> times.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.replace" title="datafusion.functions.replace"><code class="xref py py-obj docutils literal notranslate"><span class="pre">replace</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.replace" title="datafusion.functions.replace"><code class="xref py py-obj docutils literal notranslate"><span class="pre">replace</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Replaces all occurrences of <code class="docutils literal notranslate"><span class="pre">from_val</span></code> with <code class="docutils literal notranslate"><span class="pre">to_val</span></code> in the <code class="docutils literal notranslate"><span class="pre">string</span></code>.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.reverse" title="datafusion.functions.reverse"><code class="xref py py-obj docutils literal notranslate"><span class="pre">reverse</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.reverse" title="datafusion.functions.reverse"><code class="xref py py-obj docutils literal notranslate"><span class="pre">reverse</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Reverse the string argument.</p></td>
</tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.right" title="datafusion.functions.right"><code class="xref py py-obj docutils literal notranslate"><span class="pre">right</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.right" title="datafusion.functions.right"><code class="xref py py-obj docutils literal notranslate"><span class="pre">right</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Returns the last <code class="docutils literal notranslate"><span class="pre">n</span></code> characters in the <code class="docutils literal notranslate"><span class="pre">string</span></code>.</p></td>
</tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.round" title="datafusion.functions.round"><code class="xref py py-obj docutils literal notranslate"><span class="pre">round</span></code></a>() → datafusion.expr.Expr)</p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.round" title="datafusion.functions.round"><code class="xref py py-obj docutils literal notranslate"><span class="pre">round</span></code></a>() → datafusion.expr.Expr)</p></td>
<td><p>Round the argument to the nearest integer.</p></td>
</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.functions.row_number" title="datafusion.functions.row_number"><code class="xref py py-obj docutils literal notranslate"><span class="pre">row_number</span></code></a>(→ datafusion.expr.Expr)</p></td>
+<td><p>Create a row number window function.</p></td>
+</tr>
<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.functions.rpad" title="datafusion.functions.rpad"><code class="xref py py-obj docutils literal notranslate"><span class="pre">rpad</span></code></a>(→ datafusion.expr.Expr)</p></td>
<td><p>Add right padding to a string.</p></td>
</tr>
@@ -3539,6 +3635,34 @@ detailed usage.</p>
<dd><p>Computes the sample covariance.</p>
</dd></dl>
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.functions.cume_dist">
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">cume_dist</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/i [...]
+<dd><p>Create a cumulative distribution window function.</p>
+<p>This window function is similar to <a class="reference internal" href="#datafusion.functions.rank" title="datafusion.functions.rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">rank()</span></code></a> except that the returned values
+are the ratio of the row number to the total numebr of rows. Here is an example of a
+dataframe with a window ordered by descending <code class="docutils literal notranslate"><span class="pre">points</span></code> and the associated
+cumulative distribution:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+-----------+</span>
+<span class="o">|</span> <span class="n">points</span> <span class="o">|</span> <span class="n">cume_dist</span> <span class="o">|</span>
+<span class="o">+--------+-----------+</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mf">0.5</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mf">0.5</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mf">0.75</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mf">1.0</span> <span class="o">|</span>
+<span class="o">+--------+-----------+</span>
+</pre></div>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.current_date">
<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">current_date</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.expr.Expr</span></ [...]
@@ -3595,6 +3719,33 @@ detailed usage.</p>
<dd><p>Converts the argument from radians to degrees.</p>
</dd></dl>
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.functions.dense_rank">
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">dense_rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/ [...]
+<dd><p>Create a dense_rank window function.</p>
+<p>This window function is similar to <a class="reference internal" href="#datafusion.functions.rank" title="datafusion.functions.rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">rank()</span></code></a> except that the returned values
+will be consecutive. Here is an example of a dataframe with a window ordered by
+descending <code class="docutils literal notranslate"><span class="pre">points</span></code> and the associated dense rank:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+------------+</span>
+<span class="o">|</span> <span class="n">points</span> <span class="o">|</span> <span class="n">dense_rank</span> <span class="o">|</span>
+<span class="o">+--------+------------+</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span>
+<span class="o">+--------+------------+</span>
+</pre></div>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.digest">
<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">digest</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusio [...]
@@ -3638,7 +3789,7 @@ blake2b, and blake3.</p>
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.first_value">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">first_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafu [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">first_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafu [...]
<dd><p>Returns the first value in a group of values.</p>
</dd></dl>
@@ -3699,10 +3850,45 @@ characters to lowercase.</p>
<dd><p>Returns true if a given number is +0.0 or -0.0 otherwise returns false.</p>
</dd></dl>
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.functions.lag">
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">lag</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.exp [...]
+<dd><p>Create a lag window function.</p>
+<p>Lag operation will return the argument that is in the previous shift_offset-th row
+in the partition. For example <code class="docutils literal notranslate"><span class="pre">lag(col("b"),</span> <span class="pre">shift_offset=3,</span> <span class="pre">default_value=5)</span></code>
+will return the 3rd previous value in column <code class="docutils literal notranslate"><span class="pre">b</span></code>. At the beginnig of the
+partition, where no values can be returned it will return the default value of 5.</p>
+<p>Here is an example of both the <code class="docutils literal notranslate"><span class="pre">lag</span></code> and <a class="reference internal" href="#datafusion.functions.lead" title="datafusion.functions.lead"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.lead()</span></code></a>
+functions on a simple DataFrame:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+------+-----+</span>
+<span class="o">|</span> <span class="n">points</span> <span class="o">|</span> <span class="n">lead</span> <span class="o">|</span> <span class="n">lag</span> <span class="o">|</span>
+<span class="o">+--------+------+-----+</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="o">|</span> <span class="mi">50</span> <span class="o">|</span>
+<span class="o">+--------+------+-----+</span>
+</pre></div>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>arg</strong> – Value to return</p></li>
+<li><p><strong>shift_offset</strong> – Number of rows before the current row.</p></li>
+<li><p><strong>default_value</strong> – Value to return if shift_offet row does not exist.</p></li>
+<li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.last_value">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">last_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafus [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">last_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafus [...]
<dd><p>Returns the last value in a group of values.</p>
+<p>To set parameters on this expression, use <code class="docutils literal notranslate"><span class="pre">.order_by()</span></code>, <code class="docutils literal notranslate"><span class="pre">.distinct()</span></code>,
+<code class="docutils literal notranslate"><span class="pre">.filter()</span></code>, or <code class="docutils literal notranslate"><span class="pre">.null_treatment()</span></code>.</p>
</dd></dl>
<dl class="py function">
@@ -3711,6 +3897,41 @@ characters to lowercase.</p>
<dd><p>Returns the least common multiple.</p>
</dd></dl>
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.functions.lead">
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">lead</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.ex [...]
+<dd><p>Create a lead window function.</p>
+<p>Lead operation will return the argument that is in the next shift_offset-th row in
+the partition. For example <code class="docutils literal notranslate"><span class="pre">lead(col("b"),</span> <span class="pre">shift_offset=3,</span> <span class="pre">default_value=5)</span></code> will
+return the 3rd following value in column <code class="docutils literal notranslate"><span class="pre">b</span></code>. At the end of the partition, where
+no futher values can be returned it will return the default value of 5.</p>
+<p>Here is an example of both the <code class="docutils literal notranslate"><span class="pre">lead</span></code> and <a class="reference internal" href="#datafusion.functions.lag" title="datafusion.functions.lag"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.lag()</span></code></a>
+functions on a simple DataFrame:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+------+-----+</span>
+<span class="o">|</span> <span class="n">points</span> <span class="o">|</span> <span class="n">lead</span> <span class="o">|</span> <span class="n">lag</span> <span class="o">|</span>
+<span class="o">+--------+------+-----+</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="o">|</span> <span class="mi">50</span> <span class="o">|</span>
+<span class="o">+--------+------+-----+</span>
+</pre></div>
+</div>
+<p>To set window function parameters use the window builder approach described in the
+ref:<cite>_window_functions</cite> online documentation.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>arg</strong> – Value to return</p></li>
+<li><p><strong>shift_offset</strong> – Number of rows following the current row.</p></li>
+<li><p><strong>default_value</strong> – Value to return if shift_offet row does not exist.</p></li>
+<li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.left">
<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">left</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">string</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion [...]
@@ -4011,7 +4232,7 @@ truncated (on the right).</p>
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.named_struct">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">named_struct</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name_pairs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"> [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">named_struct</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name_pairs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><span class="pre">tuple</span><span class="p [...]
<dd><p>Returns a struct with the given names and arguments pairs.</p>
</dd></dl>
@@ -4028,6 +4249,37 @@ truncated (on the right).</p>
<p>This will use the same value for all instances of now() in same statement.</p>
</dd></dl>
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.functions.ntile">
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">ntile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">groups</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span cla [...]
+<dd><p>Create a n-tile window function.</p>
+<p>This window function orders the window frame into a give number of groups based on
+the ordering criteria. It then returns which group the current row is assigned to.
+Here is an example of a dataframe with a window ordered by descending <code class="docutils literal notranslate"><span class="pre">points</span></code>
+and the associated n-tile function:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+-------+</span>
+<span class="o">|</span> <span class="n">points</span> <span class="o">|</span> <span class="n">ntile</span> <span class="o">|</span>
+<span class="o">+--------+-------+</span>
+<span class="o">|</span> <span class="mi">120</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">80</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">60</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">40</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span>
+<span class="o">+--------+-------+</span>
+</pre></div>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>groups</strong> – Number of groups for the n-tile to be divided into.</p></li>
+<li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.nullif">
<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">nullif</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expr1</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusio [...]
@@ -4055,6 +4307,34 @@ truncated (on the right).</p>
extends for <code class="docutils literal notranslate"><span class="pre">length</span></code> characters with new substring.</p>
</dd></dl>
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.functions.percent_rank">
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">percent_rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../exp [...]
+<dd><p>Create a percent_rank window function.</p>
+<p>This window function is similar to <a class="reference internal" href="#datafusion.functions.rank" title="datafusion.functions.rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">rank()</span></code></a> except that the returned values
+are the percentage from 0.0 to 1.0 from first to last. Here is an example of a
+dataframe with a window ordered by descending <code class="docutils literal notranslate"><span class="pre">points</span></code> and the associated percent
+rank:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+--------------+</span>
+<span class="o">|</span> <span class="n">points</span> <span class="o">|</span> <span class="n">percent_rank</span> <span class="o">|</span>
+<span class="o">+--------+--------------+</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mf">0.0</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mf">0.0</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mf">0.666667</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mf">1.0</span> <span class="o">|</span>
+<span class="o">+--------+--------------+</span>
+</pre></div>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.pi">
<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">pi</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.expr.Expr</span></a></span>< [...]
@@ -4092,6 +4372,37 @@ extends for <code class="docutils literal notranslate"><span class="pre">length<
<dd><p>Create a list of values in the range between start and stop.</p>
</dd></dl>
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.functions.rank">
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/index. [...]
+<dd><p>Create a rank window function.</p>
+<p>Returns the rank based upon the window order. Consecutive equal values will receive
+the same rank, but the next different value will not be consecutive but rather the
+number of rows that preceed it plus one. This is similar to Olympic medals. If two
+people tie for gold, the next place is bronze. There would be no silver medal. Here
+is an example of a dataframe with a window ordered by descending <code class="docutils literal notranslate"><span class="pre">points</span></code> and the
+associated rank.</p>
+<p>You should set <code class="docutils literal notranslate"><span class="pre">order_by</span></code> to produce meaningful results:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+------+</span>
+<span class="o">|</span> <span class="n">points</span> <span class="o">|</span> <span class="n">rank</span> <span class="o">|</span>
+<span class="o">+--------+------+</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mi">4</span> <span class="o">|</span>
+<span class="o">+--------+------+</span>
+</pre></div>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.regexp_like">
<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">regexp_like</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">string</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">dat [...]
@@ -4207,6 +4518,32 @@ decimal places. You can specify a negative number of decimal places. For example
<code class="docutils literal notranslate"><span class="pre">round(lit(125.2345),</span> <span class="pre">lit(-2))</span></code> would yield a value of <code class="docutils literal notranslate"><span class="pre">100.0</span></code>.</p>
</dd></dl>
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.functions.row_number">
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">row_number</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/ [...]
+<dd><p>Create a row number window function.</p>
+<p>Returns the row number of the window function.</p>
+<p>Here is an example of the <code class="docutils literal notranslate"><span class="pre">row_number</span></code> on a simple DataFrame:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+------------+</span>
+<span class="o">|</span> <span class="n">points</span> <span class="o">|</span> <span class="n">row</span> <span class="n">number</span> <span class="o">|</span>
+<span class="o">+--------+------------+</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mi">4</span> <span class="o">|</span>
+<span class="o">+--------+------------+</span>
+</pre></div>
+</div>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+</ul>
+</dd>
+</dl>
+</dd></dl>
+
<dl class="py function">
<dt class="sig sig-object py" id="datafusion.functions.rpad">
<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">rpad</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">string</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion [...]
@@ -4455,6 +4792,12 @@ detailed usage.</p>
<dt class="sig sig-object py" id="datafusion.functions.window">
<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">window</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">args</span></span><span class="p"><s [...]
<dd><p>Creates a new Window function expression.</p>
+<p>This interface will soon be deprecated. Instead of using this interface,
+users should call the window functions directly. For example, to perform a
+lag use:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">functions</span><span class="o">.</span><span class="n">lag</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">"a"</span><span class="p">))</span><span class="o">.</span><span class="n">partition_by</span><span class="p">(</span><span [...]
+</pre></div>
+</div>
</dd></dl>
</section>
diff --git a/autoapi/datafusion/index.html b/autoapi/datafusion/index.html
index 9bd509e..0da78ee 100644
--- a/autoapi/datafusion/index.html
+++ b/autoapi/datafusion/index.html
@@ -752,6 +752,33 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.Expr.distinct">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.distinct()
+ </span>
+ </code>
+ </a>
+ </li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.Expr.filter">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.filter()
+ </span>
+ </code>
+ </a>
+ </li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.Expr.is_not_null">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.is_not_null()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h4 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.Expr.is_null">
<code class="docutils literal notranslate">
@@ -770,6 +797,33 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.Expr.null_treatment">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.null_treatment()
+ </span>
+ </code>
+ </a>
+ </li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.Expr.order_by">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.order_by()
+ </span>
+ </code>
+ </a>
+ </li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.Expr.partition_by">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.partition_by()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h4 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.Expr.python_value">
<code class="docutils literal notranslate">
@@ -842,6 +896,15 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.Expr.window_frame">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ Expr.window_frame()
+ </span>
+ </code>
+ </a>
+ </li>
<li class="toc-h4 nav-item toc-entry">
<a class="reference internal nav-link" href="#datafusion.Expr.__radd__">
<code class="docutils literal notranslate">
@@ -1838,6 +1901,30 @@ aggregation or window function calls.</p>
<p>This name will not include any CAST expressions.</p>
</dd></dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.Expr.distinct">
+<span class="sig-name descname"><span class="pre">distinct</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">ExprFuncBuilder</span></span></span><a class="headerlink" href="#datafusion.Expr.distinct" title="Link to this definition">¶</a></dt>
+<dd><p>Only evaluate distinct values for an aggregate function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.Expr.filter">
+<span class="sig-name descname"><span class="pre">filter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">filter</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.Expr" title="datafusion.Expr"><span class="pre">Expr</span></a></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span [...]
+<dd><p>Filter an aggregate function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.Expr.is_not_null">
+<span class="sig-name descname"><span class="pre">is_not_null</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#datafusion.Expr" title="datafusion.Expr"><span class="pre">Expr</span></a></span></span><a class="headerlink" href="#datafusion.Expr.is_not_null" title="Link to this definition">¶</a></dt>
+<dd><p>Returns <code class="docutils literal notranslate"><span class="pre">True</span></code> if this expression is not null.</p>
+</dd></dl>
+
<dl class="py method">
<dt class="sig sig-object py" id="datafusion.Expr.is_null">
<span class="sig-name descname"><span class="pre">is_null</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><a class="reference internal" href="#datafusion.Expr" title="datafusion.Expr"><span class="pre">Expr</span></a></span></span><a class="headerlink" href="#datafusion.Expr.is_null" title="Link to this definition">¶</a></dt>
@@ -1851,6 +1938,33 @@ aggregation or window function calls.</p>
<p><code class="docutils literal notranslate"><span class="pre">value</span></code> must be a valid PyArrow scalar value or easily castable to one.</p>
</dd></dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.Expr.null_treatment">
+<span class="sig-name descname"><span class="pre">null_treatment</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">null_treatment</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">datafusion.common.NullTreatment</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><sp [...]
+<dd><p>Set the treatment for <code class="docutils literal notranslate"><span class="pre">null</span></code> values for a window or aggregate function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.Expr.order_by">
+<span class="sig-name descname"><span class="pre">order_by</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">exprs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.Expr" title="datafusion.Expr"><span class="pre">Expr</span></a></span></em><span class="sig-paren">)</span> <span class="sig-re [...]
+<dd><p>Set the ordering for a window or aggregate function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.Expr.partition_by">
+<span class="sig-name descname"><span class="pre">partition_by</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.Expr" title="datafusion.Expr"><span class="pre">Expr</span></a></span></em><span class="sig-paren">)</span> <span cl [...]
+<dd><p>Set the partitioning for a window function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
<dl class="py method">
<dt class="sig sig-object py" id="datafusion.Expr.python_value">
<span class="sig-name descname"><span class="pre">python_value</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">→</span> <span class="sig-return-typehint"><span class="pre">Any</span></span></span><a class="headerlink" href="#datafusion.Expr.python_value" title="Link to this definition">¶</a></dt>
@@ -1927,6 +2041,15 @@ SqlType Enum which this expression represents.</p>
<p>Ex: <code class="docutils literal notranslate"><span class="pre">IsNotNull</span></code>, <code class="docutils literal notranslate"><span class="pre">Literal</span></code>, <code class="docutils literal notranslate"><span class="pre">BinaryExpr</span></code>, etc</p>
</dd></dl>
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.Expr.window_frame">
+<span class="sig-name descname"><span class="pre">window_frame</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">window_frame</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.WindowFrame" title="datafusion.WindowFrame"><span class="pre">WindowFrame</span></a></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class [...]
+<dd><p>Set the frame fora window function.</p>
+<p>This function will create an <code class="xref py py-class docutils literal notranslate"><span class="pre">ExprFuncBuilder</span></code> that can be used to
+set parameters for either window or aggregate functions. If used on any other
+type of expression, an error will be generated when <code class="docutils literal notranslate"><span class="pre">build()</span></code> is called.</p>
+</dd></dl>
+
<dl class="py attribute">
<dt class="sig sig-object py" id="datafusion.Expr.__radd__">
<span class="sig-name descname"><span class="pre">__radd__</span></span><a class="headerlink" href="#datafusion.Expr.__radd__" title="Link to this definition">¶</a></dt>
@@ -2508,7 +2631,7 @@ automatically created.</p>
<dl class="py class">
<dt class="sig sig-object py" id="datafusion.WindowFrame">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.</span></span><span class="sig-name descname"><span class="pre">WindowFrame</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">units</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-p [...]
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.</span></span><span class="sig-name descname"><span class="pre">WindowFrame</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">units</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-p [...]
<dd><p>Defines a window frame for performing window operations.</p>
<p>Construct a window frame using the given parameters.</p>
<dl class="field-list simple">
diff --git a/autoapi/datafusion/udf/index.html b/autoapi/datafusion/udf/index.html
index dcc70f5..6e36743 100644
--- a/autoapi/datafusion/udf/index.html
+++ b/autoapi/datafusion/udf/index.html
@@ -735,7 +735,7 @@ function.</p></li>
<dl class="py class">
<dt class="sig sig-object py" id="datafusion.udf.Volatility">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.udf.</span></span><span class="sig-name descname"><span class="pre">Volatility</span></span><a class="headerlink" href="#datafusion.udf.Volatility" title="Link to this definition">¶</a></dt>
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.udf.</span></span><span class="sig-name descname"><span class="pre">Volatility</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">args</span></span></em>, <em class="sig-param"><span class="o"><span class="pre">**</span></span><span class="n">< [...]
<dd><p>Bases: <code class="xref py py-obj docutils literal notranslate"><span class="pre">enum.Enum</span></code></p>
<p>Defines how stable or volatile a function is.</p>
<p>When setting the volatility of a function, you can either pass this
diff --git a/genindex.html b/genindex.html
index 454ab70..f7bc94b 100644
--- a/genindex.html
+++ b/genindex.html
@@ -878,6 +878,8 @@
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.CrossJoin">CrossJoin (in module datafusion.expr)</a>
</li>
<li><a href="autoapi/datafusion/context/index.html#datafusion.context.SessionContext.ctx">ctx (datafusion.context.SessionContext attribute)</a>
+</li>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.cume_dist">cume_dist() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.current_date">current_date() (in module datafusion.functions)</a>
</li>
@@ -1015,6 +1017,8 @@
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.decode">decode() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.degrees">degrees() (in module datafusion.functions)</a>
+</li>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.dense_rank">dense_rank() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/context/index.html#datafusion.context.SessionContext.deregister_table">deregister_table() (datafusion.context.SessionContext method)</a>
</li>
@@ -1039,7 +1043,13 @@
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Distinct">Distinct (in module datafusion.expr)</a>
</li>
<li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame.distinct">distinct() (datafusion.dataframe.DataFrame method)</a>
+
+ <ul>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Expr.distinct">(datafusion.Expr method)</a>
</li>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.distinct">(datafusion.expr.Expr method)</a>
+</li>
+ </ul></li>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.DropTable">DropTable (in module datafusion.expr)</a>
</li>
</ul></td>
@@ -1113,7 +1123,13 @@
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Filter">Filter (in module datafusion.expr)</a>
</li>
<li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame.filter">filter() (datafusion.dataframe.DataFrame method)</a>
+
+ <ul>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Expr.filter">(datafusion.Expr method)</a>
+</li>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.filter">(datafusion.expr.Expr method)</a>
</li>
+ </ul></li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.find_in_set">find_in_set() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.first_value">first_value() (in module datafusion.functions)</a>
@@ -1121,11 +1137,11 @@
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.flatten">flatten() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.floor">floor() (in module datafusion.functions)</a>
-</li>
- <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.WindowFrameBound.frame_bound">frame_bound (datafusion.expr.WindowFrameBound attribute)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.WindowFrameBound.frame_bound">frame_bound (datafusion.expr.WindowFrameBound attribute)</a>
+</li>
<li><a href="autoapi/datafusion/context/index.html#datafusion.context.SessionContext.from_arrow">from_arrow() (datafusion.context.SessionContext method)</a>
</li>
<li><a href="autoapi/datafusion/context/index.html#datafusion.context.SessionContext.from_arrow_table">from_arrow_table() (datafusion.context.SessionContext method)</a>
@@ -1208,14 +1224,20 @@
</li>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.WindowFrameBound.is_following">is_following() (datafusion.expr.WindowFrameBound method)</a>
</li>
- <li><a href="autoapi/datafusion/index.html#datafusion.Expr.is_null">is_null() (datafusion.Expr method)</a>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Expr.is_not_null">is_not_null() (datafusion.Expr method)</a>
<ul>
- <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.is_null">(datafusion.expr.Expr method)</a>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.is_not_null">(datafusion.expr.Expr method)</a>
</li>
</ul></li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Expr.is_null">is_null() (datafusion.Expr method)</a>
+
+ <ul>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.is_null">(datafusion.expr.Expr method)</a>
+</li>
+ </ul></li>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.WindowFrameBound.is_preceding">is_preceding() (datafusion.expr.WindowFrameBound method)</a>
</li>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.WindowFrameBound.is_unbounded">is_unbounded() (datafusion.expr.WindowFrameBound method)</a>
@@ -1274,9 +1296,13 @@
<h2 id="L">L</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.lag">lag() (in module datafusion.functions)</a>
+</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.last_value">last_value() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.lcm">lcm() (in module datafusion.functions)</a>
+</li>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.lead">lead() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.left">left() (in module datafusion.functions)</a>
</li>
@@ -1321,11 +1347,11 @@
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.list_push_back">list_push_back() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.list_push_front">list_push_front() (in module datafusion.functions)</a>
-</li>
- <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.list_remove">list_remove() (in module datafusion.functions)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.list_remove">list_remove() (in module datafusion.functions)</a>
+</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.list_remove_all">list_remove_all() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.list_remove_n">list_remove_n() (in module datafusion.functions)</a>
@@ -1457,10 +1483,10 @@
</ul></li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.nanvl">nanvl() (in module datafusion.functions)</a>
</li>
- </ul></td>
- <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Negative">Negative (in module datafusion.expr)</a>
</li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="autoapi/datafusion/record_batch/index.html#datafusion.record_batch.RecordBatchStream.next">next() (datafusion.record_batch.RecordBatchStream method)</a>
<ul>
@@ -1471,6 +1497,14 @@
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.now">now() (in module datafusion.functions)</a>
</li>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.ntile">ntile() (in module datafusion.functions)</a>
+</li>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Expr.null_treatment">null_treatment() (datafusion.Expr method)</a>
+
+ <ul>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.null_treatment">(datafusion.expr.Expr method)</a>
+</li>
+ </ul></li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.nullif">nullif() (in module datafusion.functions)</a>
</li>
</ul></td>
@@ -1491,8 +1525,14 @@
</ul></li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.order_by">order_by() (in module datafusion.functions)</a>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Expr.order_by">order_by() (datafusion.Expr method)</a>
+
+ <ul>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.order_by">(datafusion.expr.Expr method)</a>
</li>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.order_by">(in module datafusion.functions)</a>
+</li>
+ </ul></li>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.CaseBuilder.otherwise">otherwise() (datafusion.expr.CaseBuilder method)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.overlay">overlay() (in module datafusion.functions)</a>
@@ -1503,18 +1543,26 @@
<h2 id="P">P</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Expr.partition_by">partition_by() (datafusion.Expr method)</a>
+
+ <ul>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.partition_by">(datafusion.expr.Expr method)</a>
+</li>
+ </ul></li>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Partitioning">Partitioning (in module datafusion.expr)</a>
+</li>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.percent_rank">percent_rank() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.pi">pi() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Placeholder">Placeholder (in module datafusion.expr)</a>
</li>
<li><a href="autoapi/datafusion/substrait/index.html#datafusion.substrait.Plan">Plan (class in datafusion.substrait)</a>
-</li>
- <li><a href="autoapi/datafusion/substrait/index.html#datafusion.substrait.Plan.plan_internal">plan_internal (datafusion.substrait.Plan attribute)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
+ <li><a href="autoapi/datafusion/substrait/index.html#datafusion.substrait.Plan.plan_internal">plan_internal (datafusion.substrait.Plan attribute)</a>
+</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.pow">pow() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.power">power() (in module datafusion.functions)</a>
@@ -1540,6 +1588,8 @@
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.random">random() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.range">range() (in module datafusion.functions)</a>
+</li>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.rank">rank() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/record_batch/index.html#datafusion.record_batch.RecordBatchStream.rbs">rbs (datafusion.record_batch.RecordBatchStream attribute)</a>
@@ -1656,6 +1706,8 @@
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.right">right() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.round">round() (in module datafusion.functions)</a>
+</li>
+ <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.row_number">row_number() (in module datafusion.functions)</a>
</li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.rpad">rpad() (in module datafusion.functions)</a>
</li>
@@ -1971,6 +2023,12 @@
<ul>
<li><a href="autoapi/datafusion/index.html#datafusion.WindowFrame.window_frame">(datafusion.WindowFrame attribute)</a>
+</li>
+ </ul></li>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Expr.window_frame">window_frame() (datafusion.Expr method)</a>
+
+ <ul>
+ <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.window_frame">(datafusion.expr.Expr method)</a>
</li>
</ul></li>
<li><a href="autoapi/datafusion/index.html#datafusion.WindowFrame">WindowFrame (class in datafusion)</a>
diff --git a/objects.inv b/objects.inv
index 1e0a607..b6981eb 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/searchindex.js b/searchindex.js
index 8e43f74..567cd78 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"API Reference": [[13, null]], "Aggregation": [[17, null]], "Arrow": [[28, null]], "Attributes": [[3, "attributes"], [5, "attributes"], [9, "attributes"], [12, "attributes"]], "Avro": [[29, null]], "Basic Operations": [[18, null]], "Boolean": [[19, "boolean"]], "CSV": [[30, null]], "Classes": [[0, "classes"], [1, "classes"], [2, "classes"], [3, "classes"], [5, "classes"], [6, "classes"], [8, "classes"], [10, "classes"], [11, "classes"], [12, "classes"]], "C [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"API Reference": [[13, null]], "Aggregate Functions": [[25, "aggregate-functions"]], "Aggregation": [[17, null]], "Arrow": [[28, null]], "Attributes": [[3, "attributes"], [5, "attributes"], [9, "attributes"], [12, "attributes"]], "Available Functions": [[25, "available-functions"]], "Avro": [[29, null]], "Basic Operations": [[18, null]], "Boolean": [[19, "boolean"]], "CSV": [[30, null]], "Classes": [[0, "classes"], [1, "classes"], [2, "classes"], [3, "class [...]
\ No newline at end of file
diff --git a/user-guide/common-operations/aggregations.html b/user-guide/common-operations/aggregations.html
index 48f15c0..939fea7 100644
--- a/user-guide/common-operations/aggregations.html
+++ b/user-guide/common-operations/aggregations.html
@@ -362,7 +362,7 @@
<div>
<section id="aggregation">
-<h1>Aggregation<a class="headerlink" href="#aggregation" title="Link to this heading">¶</a></h1>
+<span id="id1"></span><h1>Aggregation<a class="headerlink" href="#aggregation" title="Link to this heading">¶</a></h1>
<p>An aggregate or aggregation is a function where the values of multiple rows are processed together to form a single summary value.
For performing an aggregation, DataFusion provides the <a class="reference internal" href="../../autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame.aggregate" title="datafusion.dataframe.DataFrame.aggregate"><code class="xref py py-func docutils literal notranslate"><span class="pre">aggregate()</span></code></a></p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="kn">from</span> <span class="nn">datafusion</span> <span class="kn">import</span> <span class="n">SessionContext</span>
@@ -400,7 +400,7 @@ For performing an aggregation, DataFusion provides the <a class="reference inter
<span class="o">+----------------------------+--------------------------+------------------------------------------------+</span>
<span class="o">|</span> <span class="n">approx_distinct</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">c</span><span class="p">)</span> <span class="o">|</span> <span class="n">approx_median</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">d</span><span class="p">)</span> <span class="o">|</span> <span class="n">approx_percentile_cont</span><span class="p">(</span><span class="n">foo_ [...]
<span class="o">+----------------------------+--------------------------+------------------------------------------------+</span>
-<span class="o">|</span> <span class="mi">8</span> <span class="o">|</span> <span class="mf">0.4324629160344956</span> <span class="o">|</span> <span class="mf">0.4324629160344956</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">8</span> <span class="o">|</span> <span class="mf">0.4514285223495125</span> <span class="o">|</span> <span class="mf">0.4514285223495125</span> <span class="o">|</span>
<span class="o">+----------------------------+--------------------------+------------------------------------------------+</span>
</pre></div>
</div>
@@ -409,28 +409,28 @@ the <code class="code docutils literal notranslate"><span class="pre">group_by</
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">aggregate</span><span class="p">([</span><span class="n">col_a</span><span class="p">],</span> <span class="p">[</span><span class="n">f</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">col_c</s [...]
<span class="n">Out</span><span class="p">[</span><span class="mi">12</span><span class="p">]:</span>
<span class="n">DataFrame</span><span class="p">()</span>
-<span class="o">+-----+----------------+--------------------+----------------------+</span>
-<span class="o">|</span> <span class="n">a</span> <span class="o">|</span> <span class="nb">sum</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">c</span><span class="p">)</span> <span class="o">|</span> <span class="nb">max</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">d</span><span class="p">)</span> <span class="o">|</span> <span class="nb">min</span><span class="p">(</span><s [...]
-<span class="o">+-----+----------------+--------------------+----------------------+</span>
-<span class="o">|</span> <span class="n">foo</span> <span class="o">|</span> <span class="mi">255</span> <span class="o">|</span> <span class="mf">0.9099653871772082</span> <span class="o">|</span> <span class="mf">0.054149397012223544</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">bar</span> <span class="o">|</span> <span class="mi">112</span> <span class="o">|</span> <span class="mf">0.5139738583272621</span> <span class="o">|</span> <span class="mf">0.4186091425839046</span> <span class="o">|</span>
-<span class="o">+-----+----------------+--------------------+----------------------+</span>
+<span class="o">+-----+----------------+--------------------+---------------------+</span>
+<span class="o">|</span> <span class="n">a</span> <span class="o">|</span> <span class="nb">sum</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">c</span><span class="p">)</span> <span class="o">|</span> <span class="nb">max</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">d</span><span class="p">)</span> <span class="o">|</span> <span class="nb">min</span><span class="p">(</span><s [...]
+<span class="o">+-----+----------------+--------------------+---------------------+</span>
+<span class="o">|</span> <span class="n">foo</span> <span class="o">|</span> <span class="mi">126</span> <span class="o">|</span> <span class="mf">0.7971995457368901</span> <span class="o">|</span> <span class="mf">0.23074184540083675</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">bar</span> <span class="o">|</span> <span class="mi">193</span> <span class="o">|</span> <span class="mf">0.4632512699705238</span> <span class="o">|</span> <span class="mf">0.03788722141121503</span> <span class="o">|</span>
+<span class="o">+-----+----------------+--------------------+---------------------+</span>
</pre></div>
</div>
<p>More than one column can be used for grouping</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">13</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">aggregate</span><span class="p">([</span><span class="n">col_a</span><span class="p">,</span> <span class="n">col_b</span><span class="p">],</span> <span class="p">[</span><span class="n">f</span><span class="o">.</span><span class="n">sum</ [...]
<span class="n">Out</span><span class="p">[</span><span class="mi">13</span><span class="p">]:</span>
<span class="n">DataFrame</span><span class="p">()</span>
-<span class="o">+-----+-------+----------------+----------------------+----------------------+</span>
-<span class="o">|</span> <span class="n">a</span> <span class="o">|</span> <span class="n">b</span> <span class="o">|</span> <span class="nb">sum</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">c</span><span class="p">)</span> <span class="o">|</span> <span class="nb">max</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">d</span><span class="p">)</span> <span class="o">|</spa [...]
-<span class="o">+-----+-------+----------------+----------------------+----------------------+</span>
-<span class="o">|</span> <span class="n">bar</span> <span class="o">|</span> <span class="n">three</span> <span class="o">|</span> <span class="mi">59</span> <span class="o">|</span> <span class="mf">0.5139738583272621</span> <span class="o">|</span> <span class="mf">0.5139738583272621</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">foo</span> <span class="o">|</span> <span class="n">three</span> <span class="o">|</span> <span class="mi">26</span> <span class="o">|</span> <span class="mf">0.054149397012223544</span> <span class="o">|</span> <span class="mf">0.054149397012223544</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">foo</span> <span class="o">|</span> <span class="n">one</span> <span class="o">|</span> <span class="mi">151</span> <span class="o">|</span> <span class="mf">0.9047266102733398</span> <span class="o">|</span> <span class="mf">0.7511496845644656</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">bar</span> <span class="o">|</span> <span class="n">one</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="mf">0.4251059153933998</span> <span class="o">|</span> <span class="mf">0.4251059153933998</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">foo</span> <span class="o">|</span> <span class="n">two</span> <span class="o">|</span> <span class="mi">78</span> <span class="o">|</span> <span class="mf">0.9099653871772082</span> <span class="o">|</span> <span class="mf">0.188493695424385</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">bar</span> <span class="o">|</span> <span class="n">two</span> <span class="o">|</span> <span class="mi">51</span> <span class="o">|</span> <span class="mf">0.4186091425839046</span> <span class="o">|</span> <span class="mf">0.4186091425839046</span> <span class="o">|</span>
-<span class="o">+-----+-------+----------------+----------------------+----------------------+</span>
+<span class="o">+-----+-------+----------------+---------------------+---------------------+</span>
+<span class="o">|</span> <span class="n">a</span> <span class="o">|</span> <span class="n">b</span> <span class="o">|</span> <span class="nb">sum</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">c</span><span class="p">)</span> <span class="o">|</span> <span class="nb">max</span><span class="p">(</span><span class="n">foo_bar</span><span class="o">.</span><span class="n">d</span><span class="p">)</span> <span class="o">|</span [...]
+<span class="o">+-----+-------+----------------+---------------------+---------------------+</span>
+<span class="o">|</span> <span class="n">foo</span> <span class="o">|</span> <span class="n">one</span> <span class="o">|</span> <span class="mi">52</span> <span class="o">|</span> <span class="mf">0.4365027573437654</span> <span class="o">|</span> <span class="mf">0.23074184540083675</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">bar</span> <span class="o">|</span> <span class="n">one</span> <span class="o">|</span> <span class="mi">97</span> <span class="o">|</span> <span class="mf">0.4632512699705238</span> <span class="o">|</span> <span class="mf">0.4632512699705238</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">foo</span> <span class="o">|</span> <span class="n">two</span> <span class="o">|</span> <span class="mi">19</span> <span class="o">|</span> <span class="mf">0.7971995457368901</span> <span class="o">|</span> <span class="mf">0.45811565050620207</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">bar</span> <span class="o">|</span> <span class="n">two</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span> <span class="mf">0.04069376791530843</span> <span class="o">|</span> <span class="mf">0.04069376791530843</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">bar</span> <span class="o">|</span> <span class="n">three</span> <span class="o">|</span> <span class="mi">86</span> <span class="o">|</span> <span class="mf">0.03788722141121503</span> <span class="o">|</span> <span class="mf">0.03788722141121503</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">foo</span> <span class="o">|</span> <span class="n">three</span> <span class="o">|</span> <span class="mi">55</span> <span class="o">|</span> <span class="mf">0.6730471331408021</span> <span class="o">|</span> <span class="mf">0.6730471331408021</span> <span class="o">|</span>
+<span class="o">+-----+-------+----------------+---------------------+---------------------+</span>
</pre></div>
</div>
</section>
diff --git a/user-guide/common-operations/basic-info.html b/user-guide/common-operations/basic-info.html
index fe59ec7..f18db1a 100644
--- a/user-guide/common-operations/basic-info.html
+++ b/user-guide/common-operations/basic-info.html
@@ -384,11 +384,11 @@
<span class="o">+-----+---------+--------+--------+</span>
<span class="o">|</span> <span class="n">nrs</span> <span class="o">|</span> <span class="n">names</span> <span class="o">|</span> <span class="n">random</span> <span class="o">|</span> <span class="n">groups</span> <span class="o">|</span>
<span class="o">+-----+---------+--------+--------+</span>
-<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">791</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mi">383</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="n">java</span> <span class="o">|</span> <span class="mi">303</span> <span class="o">|</span> <span class="n">B</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="n">haskell</span> <span class="o">|</span> <span class="mi">697</span> <span class="o">|</span> <span class="n">C</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="n">go</span> <span class="o">|</span> <span class="mi">529</span> <span class="o">|</span> <span class="n">B</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">187</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mi">440</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="n">java</span> <span class="o">|</span> <span class="mi">117</span> <span class="o">|</span> <span class="n">B</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="n">haskell</span> <span class="o">|</span> <span class="mi">620</span> <span class="o">|</span> <span class="n">C</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="n">go</span> <span class="o">|</span> <span class="mi">293</span> <span class="o">|</span> <span class="n">B</span> <span class="o">|</span>
<span class="o">+-----+---------+--------+--------+</span>
</pre></div>
</div>
@@ -399,8 +399,8 @@
<span class="o">+-----+--------+--------+--------+</span>
<span class="o">|</span> <span class="n">nrs</span> <span class="o">|</span> <span class="n">names</span> <span class="o">|</span> <span class="n">random</span> <span class="o">|</span> <span class="n">groups</span> <span class="o">|</span>
<span class="o">+-----+--------+--------+--------+</span>
-<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">791</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mi">383</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">187</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mi">440</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
<span class="o">+-----+--------+--------+--------+</span>
</pre></div>
</div>
@@ -418,11 +418,11 @@ passing them to an Arrow table, and then converting them to a pandas DataFrame.<
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">to_pandas</span><span class="p">()</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">8</span><span class="p">]:</span>
<span class="n">nrs</span> <span class="n">names</span> <span class="n">random</span> <span class="n">groups</span>
-<span class="mi">0</span> <span class="mi">1</span> <span class="n">python</span> <span class="mi">791</span> <span class="n">A</span>
-<span class="mi">1</span> <span class="mi">2</span> <span class="n">ruby</span> <span class="mi">383</span> <span class="n">A</span>
-<span class="mi">2</span> <span class="mi">3</span> <span class="n">java</span> <span class="mi">303</span> <span class="n">B</span>
-<span class="mi">3</span> <span class="mi">4</span> <span class="n">haskell</span> <span class="mi">697</span> <span class="n">C</span>
-<span class="mi">4</span> <span class="mi">5</span> <span class="n">go</span> <span class="mi">529</span> <span class="n">B</span>
+<span class="mi">0</span> <span class="mi">1</span> <span class="n">python</span> <span class="mi">187</span> <span class="n">A</span>
+<span class="mi">1</span> <span class="mi">2</span> <span class="n">ruby</span> <span class="mi">440</span> <span class="n">A</span>
+<span class="mi">2</span> <span class="mi">3</span> <span class="n">java</span> <span class="mi">117</span> <span class="n">B</span>
+<span class="mi">3</span> <span class="mi">4</span> <span class="n">haskell</span> <span class="mi">620</span> <span class="n">C</span>
+<span class="mi">4</span> <span class="mi">5</span> <span class="n">go</span> <span class="mi">293</span> <span class="n">B</span>
</pre></div>
</div>
<p><a class="reference internal" href="../../autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame.describe" title="datafusion.dataframe.DataFrame.describe"><code class="xref py py-func docutils literal notranslate"><span class="pre">describe()</span></code></a> shows a quick statistic summary of your data:</p>
@@ -434,11 +434,11 @@ passing them to an Arrow table, and then converting them to a pandas DataFrame.<
<span class="o">+------------+--------------------+-------+--------------------+--------+</span>
<span class="o">|</span> <span class="n">count</span> <span class="o">|</span> <span class="mf">5.0</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="mf">5.0</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">null_count</span> <span class="o">|</span> <span class="mf">0.0</span> <span class="o">|</span> <span class="mi">0</span> <span class="o">|</span> <span class="mf">0.0</span> <span class="o">|</span> <span class="mi">0</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">mean</span> <span class="o">|</span> <span class="mf">3.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">540.6</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">std</span> <span class="o">|</span> <span class="mf">1.5811388300841898</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">205.29685823217073</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
-<span class="o">|</span> <span class="nb">min</span> <span class="o">|</span> <span class="mf">1.0</span> <span class="o">|</span> <span class="n">go</span> <span class="o">|</span> <span class="mf">303.0</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
-<span class="o">|</span> <span class="nb">max</span> <span class="o">|</span> <span class="mf">5.0</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mf">791.0</span> <span class="o">|</span> <span class="n">C</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">median</span> <span class="o">|</span> <span class="mf">3.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">529.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">mean</span> <span class="o">|</span> <span class="mf">3.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">331.4</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">std</span> <span class="o">|</span> <span class="mf">1.5811388300841898</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">202.09972785731307</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
+<span class="o">|</span> <span class="nb">min</span> <span class="o">|</span> <span class="mf">1.0</span> <span class="o">|</span> <span class="n">go</span> <span class="o">|</span> <span class="mf">117.0</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="nb">max</span> <span class="o">|</span> <span class="mf">5.0</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mf">620.0</span> <span class="o">|</span> <span class="n">C</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">median</span> <span class="o">|</span> <span class="mf">3.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">293.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
<span class="o">+------------+--------------------+-------+--------------------+--------+</span>
</pre></div>
</div>
diff --git a/user-guide/common-operations/functions.html b/user-guide/common-operations/functions.html
index 9ee6fa6..4ffd5cb 100644
--- a/user-guide/common-operations/functions.html
+++ b/user-guide/common-operations/functions.html
@@ -405,7 +405,7 @@ In here we will cover some of the more popular use cases. If you want to view al
<span class="o">...</span><span class="p">:</span> <span class="s2">"pokemon.csv"</span><span class="p">,</span>
<span class="o">...</span><span class="p">:</span> <span class="p">)</span>
<span class="o">...</span><span class="p">:</span>
-<span class="n">Out</span><span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="p">(</span><span class="s1">'pokemon.csv'</span><span class="p">,</span> <span class="o"><</span><span class="n">http</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">HTTPMessage</span> <span class="n">at</span> <span class="mh">0x7f8e60f05d10</span><span class="o">></span><span class="p">)</span>
+<span class="n">Out</span><span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="p">(</span><span class="s1">'pokemon.csv'</span><span class="p">,</span> <span class="o"><</span><span class="n">http</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">HTTPMessage</span> <span class="n">at</span> <span class="mh">0x7f9afd6177d0</span><span class="o">></span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
@@ -479,16 +479,16 @@ In here we will cover some of the more popular use cases. If you want to view al
<span class="o">+--------------------------------+</span>
<span class="o">|</span> <span class="n">now</span><span class="p">()</span> <span class="o">|</span>
<span class="o">+--------------------------------+</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">30</span><span class="n">T21</span><span class="p">:</span><span class="mi">08</span><span class="p">:</span><span class="mf">40.062457495</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2024</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">02</span><span class="n">T14</span><span class="p">:</span><span class="mi">54</span><span class="p">:</span><span class="mf">44.621567833</span><span class="n">Z</span> <span class="o">|</span>
<span class="o">+--------------------------------+</span>
</pre></div>
</div>
diff --git a/user-guide/common-operations/index.html b/user-guide/common-operations/index.html
index fb0f2fe..b6d8f9e 100644
--- a/user-guide/common-operations/index.html
+++ b/user-guide/common-operations/index.html
@@ -391,7 +391,12 @@
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="aggregations.html">Aggregation</a></li>
-<li class="toctree-l1"><a class="reference internal" href="windows.html">Window Functions</a></li>
+<li class="toctree-l1"><a class="reference internal" href="windows.html">Window Functions</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="windows.html#setting-parameters">Setting Parameters</a></li>
+<li class="toctree-l2"><a class="reference internal" href="windows.html#aggregate-functions">Aggregate Functions</a></li>
+<li class="toctree-l2"><a class="reference internal" href="windows.html#available-functions">Available Functions</a></li>
+</ul>
+</li>
<li class="toctree-l1"><a class="reference internal" href="udf-and-udfa.html">User Defined Functions</a></li>
</ul>
</div>
diff --git a/user-guide/common-operations/joins.html b/user-guide/common-operations/joins.html
index 617be98..c0201da 100644
--- a/user-guide/common-operations/joins.html
+++ b/user-guide/common-operations/joins.html
@@ -452,9 +452,9 @@ values for the corresponding columns.</p>
<span class="o">+-------------+----------+----+-----------+</span>
<span class="o">|</span> <span class="n">customer_id</span> <span class="o">|</span> <span class="n">customer</span> <span class="o">|</span> <span class="nb">id</span> <span class="o">|</span> <span class="n">name</span> <span class="o">|</span>
<span class="o">+-------------+----------+----+-----------+</span>
+<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Alice</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">CityCabs</span> <span class="o">|</span>
<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">Bob</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">MetroRide</span> <span class="o">|</span>
<span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="n">Charlie</span> <span class="o">|</span> <span class="o">|</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Alice</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">CityCabs</span> <span class="o">|</span>
<span class="o">+-------------+----------+----+-----------+</span>
</pre></div>
</div>
@@ -469,8 +469,8 @@ is no match. Unmatched rows will have null values.</p>
<span class="o">+-------------+----------+----+-----------+</span>
<span class="o">|</span> <span class="n">customer_id</span> <span class="o">|</span> <span class="n">customer</span> <span class="o">|</span> <span class="nb">id</span> <span class="o">|</span> <span class="n">name</span> <span class="o">|</span>
<span class="o">+-------------+----------+----+-----------+</span>
-<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">Bob</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">MetroRide</span> <span class="o">|</span>
<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Alice</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">CityCabs</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">Bob</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">MetroRide</span> <span class="o">|</span>
<span class="o">|</span> <span class="o">|</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="n">UrbanGo</span> <span class="o">|</span>
<span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="n">Charlie</span> <span class="o">|</span> <span class="o">|</span> <span class="o">|</span>
<span class="o">+-------------+----------+----+-----------+</span>
@@ -487,8 +487,8 @@ omitting duplicates with multiple matches in the right table.</p>
<span class="o">+-------------+----------+</span>
<span class="o">|</span> <span class="n">customer_id</span> <span class="o">|</span> <span class="n">customer</span> <span class="o">|</span>
<span class="o">+-------------+----------+</span>
-<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">Bob</span> <span class="o">|</span>
<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Alice</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">Bob</span> <span class="o">|</span>
<span class="o">+-------------+----------+</span>
</pre></div>
</div>
diff --git a/user-guide/common-operations/select-and-filter.html b/user-guide/common-operations/select-and-filter.html
index c2d2676..22671c4 100644
--- a/user-guide/common-operations/select-and-filter.html
+++ b/user-guide/common-operations/select-and-filter.html
@@ -373,7 +373,7 @@
<span class="n">In</span> <span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="n">urllib</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">urlretrieve</span><span class="p">(</span><span class="s2">"https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2021-01.parquet"</span><span class="p">,</span>
<span class="o">...</span><span class="p">:</span> <span class="s2">"yellow_trip_data.parquet"</span><span class="p">)</span>
<span class="o">...</span><span class="p">:</span>
-<span class="n">Out</span><span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="p">(</span><span class="s1">'yellow_trip_data.parquet'</span><span class="p">,</span> <span class="o"><</span><span class="n">http</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">HTTPMessage</span> <span class="n">at</span> <span class="mh">0x7f8e60f47dd0</span><span class="o">></span><span class="p">)</span>
+<span class="n">Out</span><span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="p">(</span><span class="s1">'yellow_trip_data.parquet'</span><span class="p">,</span> <span class="o"><</span><span class="n">http</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">HTTPMessage</span> <span class="n">at</span> <span class="mh">0x7f9af4f0fdd0</span><span class="o">></span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">4</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
diff --git a/user-guide/common-operations/windows.html b/user-guide/common-operations/windows.html
index f81d314..5cbd49e 100644
--- a/user-guide/common-operations/windows.html
+++ b/user-guide/common-operations/windows.html
@@ -339,9 +339,51 @@
<div class="toc-item">
+<div class="tocsection onthispage pt-5 pb-3">
+ <i class="fas fa-list"></i> On this page
+</div>
<nav id="bd-toc-nav">
-
+ <ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#setting-parameters">
+ Setting Parameters
+ </a>
+ <ul class="visible nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#ordering">
+ Ordering
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#partitions">
+ Partitions
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#window-frame">
+ Window Frame
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#null-treatment">
+ Null Treatment
+ </a>
+ </li>
+ </ul>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#aggregate-functions">
+ Aggregate Functions
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#available-functions">
+ Available Functions
+ </a>
+ </li>
+</ul>
+
</nav>
</div>
@@ -362,10 +404,11 @@
<div>
<section id="window-functions">
-<h1>Window Functions<a class="headerlink" href="#window-functions" title="Link to this heading">¶</a></h1>
-<p>In this section you will learn about window functions. A window function utilizes values from one or multiple rows to
-produce a result for each individual row, unlike an aggregate function that provides a single value for multiple rows.</p>
-<p>The functionality of window functions in DataFusion is supported by the dedicated <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.window" title="datafusion.functions.window"><code class="xref py py-func docutils literal notranslate"><span class="pre">window()</span></code></a> function.</p>
+<span id="id1"></span><h1>Window Functions<a class="headerlink" href="#window-functions" title="Link to this heading">¶</a></h1>
+<p>In this section you will learn about window functions. A window function utilizes values from one or
+multiple rows to produce a result for each individual row, unlike an aggregate function that
+provides a single value for multiple rows.</p>
+<p>The window functions are availble in the <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#module-datafusion.functions" title="datafusion.functions"><code class="xref py py-mod docutils literal notranslate"><span class="pre">functions</span></code></a> module.</p>
<p>We’ll use the pokemon dataset (from Ritchie Vink) in the following examples.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="kn">import</span> <span class="nn">urllib.request</span>
@@ -380,108 +423,285 @@ produce a result for each individual row, unlike an aggregate function that prov
<span class="o">...</span><span class="p">:</span> <span class="s2">"pokemon.csv"</span><span class="p">,</span>
<span class="o">...</span><span class="p">:</span> <span class="p">)</span>
<span class="o">...</span><span class="p">:</span>
-<span class="n">Out</span><span class="p">[</span><span class="mi">5</span><span class="p">]:</span> <span class="p">(</span><span class="s1">'pokemon.csv'</span><span class="p">,</span> <span class="o"><</span><span class="n">http</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">HTTPMessage</span> <span class="n">at</span> <span class="mh">0x7f8e60fc6f10</span><span class="o">></span><span class="p">)</span>
+<span class="n">Out</span><span class="p">[</span><span class="mi">5</span><span class="p">]:</span> <span class="p">(</span><span class="s1">'pokemon.csv'</span><span class="p">,</span> <span class="o"><</span><span class="n">http</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">HTTPMessage</span> <span class="n">at</span> <span class="mh">0x7f9af4f4cd90</span><span class="o">></span><span class="p">)</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">6</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
<span class="n">In</span> <span class="p">[</span><span class="mi">7</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="s2">"pokemon.csv"</span><span class="p">)</span>
</pre></div>
</div>
-<p>Here is an example that shows how to compare each pokemons’s attack power with the average attack power in its <code class="docutils literal notranslate"><span class="pre">"Type</span> <span class="pre">1"</span></code></p>
+<p>Here is an example that shows how you can compare each pokemon’s speed to the speed of the
+previous row in the DataFrame.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span>
<span class="o">...</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Name"'</span><span class="p">),</span>
- <span class="o">...</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">),</span>
- <span class="o">...</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span>
- <span class="o">...</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span><span class="s2">"avg"</span><span class="p">,</span> <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">)],</span> <span class="n">partition_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><sp [...]
- <span class="o">...</span><span class="p">:</span> <span class="s2">"Average Attack"</span><span class="p">,</span>
- <span class="o">...</span><span class="p">:</span> <span class="p">)</span>
+ <span class="o">...</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Speed"'</span><span class="p">),</span>
+ <span class="o">...</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">lag</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Speed"'</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"Previous Speed"</span><span class="p">)</span>
<span class="o">...</span><span class="p">:</span> <span class="p">)</span>
<span class="o">...</span><span class="p">:</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">8</span><span class="p">]:</span>
<span class="n">DataFrame</span><span class="p">()</span>
-<span class="o">+------------+--------+--------------------+</span>
-<span class="o">|</span> <span class="n">Name</span> <span class="o">|</span> <span class="n">Attack</span> <span class="o">|</span> <span class="n">Average</span> <span class="n">Attack</span> <span class="o">|</span>
-<span class="o">+------------+--------+--------------------+</span>
-<span class="o">|</span> <span class="n">Jynx</span> <span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mf">50.0</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Articuno</span> <span class="o">|</span> <span class="mi">85</span> <span class="o">|</span> <span class="mf">67.5</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Farfetch</span><span class="s1">'d | 65 | 65.0 |</span>
-<span class="o">|</span> <span class="n">Jigglypuff</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span> <span class="mf">55.0</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Rattata</span> <span class="o">|</span> <span class="mi">56</span> <span class="o">|</span> <span class="mf">55.333333333333336</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Raticate</span> <span class="o">|</span> <span class="mi">81</span> <span class="o">|</span> <span class="mf">61.75</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Spearow</span> <span class="o">|</span> <span class="mi">60</span> <span class="o">|</span> <span class="mf">61.4</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Fearow</span> <span class="o">|</span> <span class="mi">90</span> <span class="o">|</span> <span class="mf">66.16666666666667</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Doduo</span> <span class="o">|</span> <span class="mi">85</span> <span class="o">|</span> <span class="mf">68.85714285714286</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Wigglytuff</span> <span class="o">|</span> <span class="mi">70</span> <span class="o">|</span> <span class="mf">69.0</span> <span class="o">|</span>
-<span class="o">+------------+--------+--------------------+</span>
+<span class="o">+---------------------------+-------+----------------+</span>
+<span class="o">|</span> <span class="n">Name</span> <span class="o">|</span> <span class="n">Speed</span> <span class="o">|</span> <span class="n">Previous</span> <span class="n">Speed</span> <span class="o">|</span>
+<span class="o">+---------------------------+-------+----------------+</span>
+<span class="o">|</span> <span class="n">Bulbasaur</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Ivysaur</span> <span class="o">|</span> <span class="mi">60</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Venusaur</span> <span class="o">|</span> <span class="mi">80</span> <span class="o">|</span> <span class="mi">60</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">VenusaurMega</span> <span class="n">Venusaur</span> <span class="o">|</span> <span class="mi">80</span> <span class="o">|</span> <span class="mi">80</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Charmander</span> <span class="o">|</span> <span class="mi">65</span> <span class="o">|</span> <span class="mi">80</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Charmeleon</span> <span class="o">|</span> <span class="mi">80</span> <span class="o">|</span> <span class="mi">65</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Charizard</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">80</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">CharizardMega</span> <span class="n">Charizard</span> <span class="n">X</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">CharizardMega</span> <span class="n">Charizard</span> <span class="n">Y</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Squirtle</span> <span class="o">|</span> <span class="mi">43</span> <span class="o">|</span> <span class="mi">100</span> <span class="o">|</span>
+<span class="o">+---------------------------+-------+----------------+</span>
</pre></div>
</div>
-<p>You can also control the order in which rows are processed by window functions by providing
+<section id="setting-parameters">
+<h2>Setting Parameters<a class="headerlink" href="#setting-parameters" title="Link to this heading">¶</a></h2>
+<section id="ordering">
+<h3>Ordering<a class="headerlink" href="#ordering" title="Link to this heading">¶</a></h3>
+<p>You can control the order in which rows are processed by window functions by providing
a list of <code class="docutils literal notranslate"><span class="pre">order_by</span></code> functions for the <code class="docutils literal notranslate"><span class="pre">order_by</span></code> parameter.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span>
<span class="o">...</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Name"'</span><span class="p">),</span>
<span class="o">...</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">),</span>
- <span class="o">...</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span>
- <span class="o">...</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span>
- <span class="o">...</span><span class="p">:</span> <span class="s2">"rank"</span><span class="p">,</span>
- <span class="o">...</span><span class="p">:</span> <span class="p">[],</span>
- <span class="o">...</span><span class="p">:</span> <span class="n">partition_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">)],</span>
- <span class="o">...</span><span class="p">:</span> <span class="n">order_by</span><span class="o">=</span><span class="p">[</span><span class="n">f</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">))],</span>
- <span class="o">...</span><span class="p">:</span> <span class="p">),</span>
- <span class="o">...</span><span class="p">:</span> <span class="s2">"rank"</span><span class="p">,</span>
- <span class="o">...</span><span class="p">:</span> <span class="p">),</span>
- <span class="o">...</span><span class="p">:</span> <span class="p">)</span>
+ <span class="o">...</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">),</span>
+ <span class="o">...</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">rank</span><span class="p">(</span>
+ <span class="o">...</span><span class="p">:</span> <span class="n">partition_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">)],</span>
+ <span class="o">...</span><span class="p">:</span> <span class="n">order_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">)</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">ascending</span><span class="o">=</span><span class="kc">True</span><span class="p">)],</span>
+ <span class="o">...</span><span class="p">:</span> <span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"rank"</span><span class="p">),</span>
+ <span class="o">...</span><span class="p">:</span> <span class="p">)</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">),</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">))</span>
<span class="o">...</span><span class="p">:</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">9</span><span class="p">]:</span>
<span class="n">DataFrame</span><span class="p">()</span>
-<span class="o">+------------+--------+------+</span>
-<span class="o">|</span> <span class="n">Name</span> <span class="o">|</span> <span class="n">Attack</span> <span class="o">|</span> <span class="n">rank</span> <span class="o">|</span>
-<span class="o">+------------+--------+------+</span>
-<span class="o">|</span> <span class="n">Jynx</span> <span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Articuno</span> <span class="o">|</span> <span class="mi">85</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Chansey</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Jigglypuff</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Meowth</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Pidgey</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Ditto</span> <span class="o">|</span> <span class="mi">48</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Lickitung</span> <span class="o">|</span> <span class="mi">55</span> <span class="o">|</span> <span class="mi">6</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Eevee</span> <span class="o">|</span> <span class="mi">55</span> <span class="o">|</span> <span class="mi">6</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Rattata</span> <span class="o">|</span> <span class="mi">56</span> <span class="o">|</span> <span class="mi">8</span> <span class="o">|</span>
-<span class="o">+------------+--------+------+</span>
+<span class="o">+------------+--------+--------+------+</span>
+<span class="o">|</span> <span class="n">Name</span> <span class="o">|</span> <span class="n">Attack</span> <span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">rank</span> <span class="o">|</span>
+<span class="o">+------------+--------+--------+------+</span>
+<span class="o">|</span> <span class="n">Metapod</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Kakuna</span> <span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Caterpie</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">3</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Weedle</span> <span class="o">|</span> <span class="mi">35</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">4</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Butterfree</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Venonat</span> <span class="o">|</span> <span class="mi">55</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">6</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Venomoth</span> <span class="o">|</span> <span class="mi">65</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">7</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Paras</span> <span class="o">|</span> <span class="mi">70</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">8</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Beedrill</span> <span class="o">|</span> <span class="mi">90</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">9</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Parasect</span> <span class="o">|</span> <span class="mi">95</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">10</span> <span class="o">|</span>
+<span class="o">+------------+--------+--------+------+</span>
+</pre></div>
+</div>
+</section>
+<section id="partitions">
+<h3>Partitions<a class="headerlink" href="#partitions" title="Link to this heading">¶</a></h3>
+<p>A window function can take a list of <code class="docutils literal notranslate"><span class="pre">partition_by</span></code> columns similar to an
+<a class="reference internal" href="aggregations.html#aggregation"><span class="std std-ref">Aggregation Function</span></a>. This will cause the window values to be evaluated
+independently for each of the partitions. In the example above, we found the rank of each
+Pokemon per <code class="docutils literal notranslate"><span class="pre">Type</span> <span class="pre">1</span></code> partitions. We can see the first couple of each partition if we do
+the following:</p>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Name"'</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">rank</span><span class="p">(</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">partition_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">)],</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">order_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">)</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">ascending</span><span class="o">=</span><span class="kc">True</span><span class="p">)],</span>
+ <span class="o">....</span><span class="p">:</span> <span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"rank"</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="p">)</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">"rank"</span><span class="p">)</span> <span class="o"><</span> <span class="n">lit</span><span class="p">(</span><span class="mi">3</span><span class="p">))</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">co [...]
+ <span class="o">....</span><span class="p">:</span>
+<span class="n">Out</span><span class="p">[</span><span class="mi">10</span><span class="p">]:</span>
+<span class="n">DataFrame</span><span class="p">()</span>
+<span class="o">+-----------+--------+----------+------+</span>
+<span class="o">|</span> <span class="n">Name</span> <span class="o">|</span> <span class="n">Attack</span> <span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">rank</span> <span class="o">|</span>
+<span class="o">+-----------+--------+----------+------+</span>
+<span class="o">|</span> <span class="n">Metapod</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Kakuna</span> <span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="n">Bug</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Dratini</span> <span class="o">|</span> <span class="mi">64</span> <span class="o">|</span> <span class="n">Dragon</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Dragonair</span> <span class="o">|</span> <span class="mi">84</span> <span class="o">|</span> <span class="n">Dragon</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Voltorb</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="n">Electric</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Magnemite</span> <span class="o">|</span> <span class="mi">35</span> <span class="o">|</span> <span class="n">Electric</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Clefairy</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span> <span class="n">Fairy</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Clefable</span> <span class="o">|</span> <span class="mi">70</span> <span class="o">|</span> <span class="n">Fairy</span> <span class="o">|</span> <span class="mi">2</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Machop</span> <span class="o">|</span> <span class="mi">80</span> <span class="o">|</span> <span class="n">Fighting</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Mankey</span> <span class="o">|</span> <span class="mi">80</span> <span class="o">|</span> <span class="n">Fighting</span> <span class="o">|</span> <span class="mi">1</span> <span class="o">|</span>
+<span class="o">+-----------+--------+----------+------+</span>
+</pre></div>
+</div>
+</section>
+<section id="window-frame">
+<h3>Window Frame<a class="headerlink" href="#window-frame" title="Link to this heading">¶</a></h3>
+<p>When using aggregate functions, the Window Frame of defines the rows over which it operates.
+If you do not specify a Window Frame, the frame will be set depending on the following
+criteria.</p>
+<ul class="simple">
+<li><p>If an <code class="docutils literal notranslate"><span class="pre">order_by</span></code> clause is set, the default window frame is defined as the rows between
+unbounded preceeding and the current row.</p></li>
+<li><p>If an <code class="docutils literal notranslate"><span class="pre">order_by</span></code> is not set, the default frame is defined as the rows betwene unbounded
+and unbounded following (the entire partition).</p></li>
+</ul>
+<p>Window Frames are defined by three parameters: unit type, starting bound, and ending bound.</p>
+<p>The unit types available are:</p>
+<ul class="simple">
+<li><p>Rows: The starting and ending boundaries are defined by the number of rows relative to the
+current row.</p></li>
+<li><p>Range: When using Range, the <code class="docutils literal notranslate"><span class="pre">order_by</span></code> clause must have exactly one term. The boundaries
+are defined bow how close the rows are to the value of the expression in the <code class="docutils literal notranslate"><span class="pre">order_by</span></code>
+parameter.</p></li>
+<li><p>Groups: A “group” is the set of all rows that have equivalent values for all terms in the
+<code class="docutils literal notranslate"><span class="pre">order_by</span></code> clause.</p></li>
+</ul>
+<p>In this example we perform a “rolling average” of the speed of the current Pokemon and the
+two preceeding rows.</p>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="kn">from</span> <span class="nn">datafusion.expr</span> <span class="kn">import</span> <span class="n">WindowFrame</span>
+
+<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Name"'</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Speed"'</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span><span class="s2">"avg"</span><span class="p">,</span>
+ <span class="o">....</span><span class="p">:</span> <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Speed"'</span><span class="p">)],</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">order_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Speed"'</span><span class="p">)],</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">window_frame</span><span class="o">=</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">"rows"</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
+ <span class="o">....</span><span class="p">:</span> <span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"Previous Speed"</span><span class="p">)</span>
+ <span class="o">....</span><span class="p">:</span> <span class="p">)</span>
+ <span class="o">....</span><span class="p">:</span>
+<span class="n">Out</span><span class="p">[</span><span class="mi">12</span><span class="p">]:</span>
+<span class="n">DataFrame</span><span class="p">()</span>
+<span class="o">+---------------------+-------+--------------------+</span>
+<span class="o">|</span> <span class="n">Name</span> <span class="o">|</span> <span class="n">Speed</span> <span class="o">|</span> <span class="n">Previous</span> <span class="n">Speed</span> <span class="o">|</span>
+<span class="o">+---------------------+-------+--------------------+</span>
+<span class="o">|</span> <span class="n">Slowpoke</span> <span class="o">|</span> <span class="mi">15</span> <span class="o">|</span> <span class="mf">15.0</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Jigglypuff</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mf">17.5</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Geodude</span> <span class="o">|</span> <span class="mi">20</span> <span class="o">|</span> <span class="mf">18.333333333333332</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Rhyhorn</span> <span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mf">21.666666666666668</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Grimer</span> <span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mf">23.333333333333332</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Paras</span> <span class="o">|</span> <span class="mi">25</span> <span class="o">|</span> <span class="mf">25.0</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Snorlax</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mf">26.666666666666668</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Lickitung</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mf">28.333333333333332</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">SlowbroMega</span> <span class="n">Slowbro</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mf">30.0</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Slowbro</span> <span class="o">|</span> <span class="mi">30</span> <span class="o">|</span> <span class="mf">30.0</span> <span class="o">|</span>
+<span class="o">+---------------------+-------+--------------------+</span>
+</pre></div>
+</div>
+</section>
+<section id="null-treatment">
+<h3>Null Treatment<a class="headerlink" href="#null-treatment" title="Link to this heading">¶</a></h3>
+<p>When using aggregate functions as window functions, it is often useful to specify how null values
+should be treated. In order to do this you need to use the builder function. In future releases
+we expect this to be simplified in the interface.</p>
+<p>One common usage for handling nulls is the case where you want to find the last value up to the
+current row. In the following example we demonstrate how setting the null treatment to ignore
+nulls will fill in with the value of the most recent non-null row. To do this, we also will set
+the window frame so that we only process up to the current row.</p>
+<p>In this example, we filter down to one specific type of Pokemon that does have some entries in
+it’s <code class="docutils literal notranslate"><span class="pre">Type</span> <span class="pre">2</span></code> column that are null.</p>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">13</span><span class="p">]:</span> <span class="kn">from</span> <span class="nn">datafusion.common</span> <span class="kn">import</span> <span class="n">NullTreatment</span>
+
+<span class="n">In</span> <span class="p">[</span><span class="mi">14</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">)</span> <span class="o">==</span> <span class="n">lit</span><span class="p">(</span><span class="s2">"Bug"</span><span class="p">))</span><span class="o">.</s [...]
+ <span class="o">....</span><span class="p">:</span> <span class="s1">'"Name"'</span><span class="p">,</span>
+ <span class="o">....</span><span class="p">:</span> <span class="s1">'"Type 2"'</span><span class="p">,</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span><span class="s2">"last_value"</span><span class="p">,</span> <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Type 2"'</span><span class="p">)])</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">window_frame</span><span class="p">(</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">"rows"</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Speed"'</span><span class="p">))</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">null_treatment</span><span class="p">(</span><span class="n">NullTreatment</span><span class="o">.</span><span class="n">IGNORE_NULLS</span><span class="p">)</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">build</span><span class="p">()</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"last_wo_null"</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span><span class="s2">"last_value"</span><span class="p">,</span> <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Type 2"'</span><span class="p">)])</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">window_frame</span><span class="p">(</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">"rows"</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Speed"'</span><span class="p">))</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">null_treatment</span><span class="p">(</span><span class="n">NullTreatment</span><span class="o">.</span><span class="n">RESPECT_NULLS</span><span class="p">)</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">build</span><span class="p">()</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"last_with_null"</span><span class="p">)</span>
+ <span class="o">....</span><span class="p">:</span> <span class="p">)</span>
+ <span class="o">....</span><span class="p">:</span>
+<span class="n">Out</span><span class="p">[</span><span class="mi">14</span><span class="p">]:</span>
+<span class="n">DataFrame</span><span class="p">()</span>
+<span class="o">+------------+--------+--------------+----------------+</span>
+<span class="o">|</span> <span class="n">Name</span> <span class="o">|</span> <span class="n">Type</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">last_wo_null</span> <span class="o">|</span> <span class="n">last_with_null</span> <span class="o">|</span>
+<span class="o">+------------+--------+--------------+----------------+</span>
+<span class="o">|</span> <span class="n">Paras</span> <span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Metapod</span> <span class="o">|</span> <span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Parasect</span> <span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Kakuna</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Caterpie</span> <span class="o">|</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Venonat</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Weedle</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Butterfree</span> <span class="o">|</span> <span class="n">Flying</span> <span class="o">|</span> <span class="n">Flying</span> <span class="o">|</span> <span class="n">Flying</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Beedrill</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Pinsir</span> <span class="o">|</span> <span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="o">|</span>
+<span class="o">+------------+--------+--------------+----------------+</span>
</pre></div>
</div>
+</section>
+</section>
+<section id="aggregate-functions">
+<h2>Aggregate Functions<a class="headerlink" href="#aggregate-functions" title="Link to this heading">¶</a></h2>
+<p>You can use any <a class="reference internal" href="aggregations.html#aggregation"><span class="std std-ref">Aggregation Function</span></a> as a window function. Currently
+aggregate functions must use the deprecated
+<a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.window" title="datafusion.functions.window"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.window()</span></code></a> API but this should be resolved in
+DataFusion 42.0 (<a class="reference external" href="https://github.com/apache/datafusion-python/issues/833">Issue Link</a>). Here
+is an example that shows how to compare each pokemons’s attack power with the average attack
+power in its <code class="docutils literal notranslate"><span class="pre">"Type</span> <span class="pre">1"</span></code> using the <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.avg" title="datafusion.functions.avg"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.avg()</span></code></a> function.</p>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">15</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Name"'</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span><span class="s2">"avg"</span><span class="p">,</span> <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Attack"'</span><span class="p">)])</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">partition_by</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">'"Type 1"'</span><span class="p">))</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">build</span><span class="p">()</span>
+ <span class="o">....</span><span class="p">:</span> <span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">"Average Attack"</span><span class="p">),</span>
+ <span class="o">....</span><span class="p">:</span> <span class="p">)</span>
+ <span class="o">....</span><span class="p">:</span>
+<span class="n">Out</span><span class="p">[</span><span class="mi">15</span><span class="p">]:</span>
+<span class="n">DataFrame</span><span class="p">()</span>
+<span class="o">+------------+--------+--------+----------------+</span>
+<span class="o">|</span> <span class="n">Name</span> <span class="o">|</span> <span class="n">Attack</span> <span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Average</span> <span class="n">Attack</span> <span class="o">|</span>
+<span class="o">+------------+--------+--------+----------------+</span>
+<span class="o">|</span> <span class="n">Jynx</span> <span class="o">|</span> <span class="mi">50</span> <span class="o">|</span> <span class="n">Ice</span> <span class="o">|</span> <span class="mf">67.5</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Articuno</span> <span class="o">|</span> <span class="mi">85</span> <span class="o">|</span> <span class="n">Ice</span> <span class="o">|</span> <span class="mf">67.5</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Farfetch</span><span class="s1">'d | 65 | Normal | 70.625 |</span>
+<span class="o">|</span> <span class="n">Jigglypuff</span> <span class="o">|</span> <span class="mi">45</span> <span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="mf">70.625</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Rattata</span> <span class="o">|</span> <span class="mi">56</span> <span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="mf">70.625</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Raticate</span> <span class="o">|</span> <span class="mi">81</span> <span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="mf">70.625</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Spearow</span> <span class="o">|</span> <span class="mi">60</span> <span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="mf">70.625</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Fearow</span> <span class="o">|</span> <span class="mi">90</span> <span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="mf">70.625</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Doduo</span> <span class="o">|</span> <span class="mi">85</span> <span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="mf">70.625</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Wigglytuff</span> <span class="o">|</span> <span class="mi">70</span> <span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="mf">70.625</span> <span class="o">|</span>
+<span class="o">+------------+--------+--------+----------------+</span>
+</pre></div>
+</div>
+</section>
+<section id="available-functions">
+<h2>Available Functions<a class="headerlink" href="#available-functions" title="Link to this heading">¶</a></h2>
<p>The possible window functions are:</p>
<ol class="arabic simple">
<li><dl class="simple">
<dt>Rank Functions</dt><dd><ul class="simple">
-<li><p>rank</p></li>
-<li><p>dense_rank</p></li>
-<li><p>row_number</p></li>
-<li><p>ntile</p></li>
+<li><p><a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.rank" title="datafusion.functions.rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.rank()</span></code></a></p></li>
+<li><p><a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.dense_rank" title="datafusion.functions.dense_rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.dense_rank()</span></code></a></p></li>
+<li><p><a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.ntile" title="datafusion.functions.ntile"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.ntile()</span></code></a></p></li>
+<li><p><a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.row_number" title="datafusion.functions.row_number"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.row_number()</span></code></a></p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Analytical Functions</dt><dd><ul class="simple">
-<li><p>cume_dist</p></li>
-<li><p>percent_rank</p></li>
-<li><p>lag</p></li>
-<li><p>lead</p></li>
-<li><p>first_value</p></li>
-<li><p>last_value</p></li>
-<li><p>nth_value</p></li>
+<li><p><a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.cume_dist" title="datafusion.functions.cume_dist"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.cume_dist()</span></code></a></p></li>
+<li><p><a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.percent_rank" title="datafusion.functions.percent_rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.percent_rank()</span></code></a></p></li>
+<li><p><a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.lag" title="datafusion.functions.lag"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.lag()</span></code></a></p></li>
+<li><p><a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.lead" title="datafusion.functions.lead"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.lead()</span></code></a></p></li>
</ul>
</dd>
</dl>
</li>
<li><dl class="simple">
<dt>Aggregate Functions</dt><dd><ul class="simple">
-<li><p>All aggregate functions can be used as window functions.</p></li>
+<li><p>All <a class="reference internal" href="aggregations.html#aggregation"><span class="std std-ref">Aggregation Functions</span></a> can be used as window functions.</p></li>
</ul>
</dd>
</dl>
</li>
</ol>
+</section>
</section>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org