You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2025/09/17 19:32:56 UTC

(datafusion-python) branch asf-site updated: Publish built docs triggered by c3a76553b755ea64b9a161e2e9d1759ba3068a96

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-python.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new a4510684 Publish built docs triggered by c3a76553b755ea64b9a161e2e9d1759ba3068a96
a4510684 is described below

commit a45106847a9cdc17e0cc7ed5e040d46063b904c7
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Wed Sep 17 19:32:50 2025 +0000

    Publish built docs triggered by c3a76553b755ea64b9a161e2e9d1759ba3068a96
---
 _sources/autoapi/datafusion/context/index.rst.txt  |  30 +-
 .../autoapi/datafusion/dataframe/index.rst.txt     | 323 ++++-----
 _sources/autoapi/datafusion/expr/index.rst.txt     |  64 +-
 .../autoapi/datafusion/functions/index.rst.txt     | 143 +++-
 _sources/autoapi/datafusion/index.rst.txt          | 268 +++----
 .../autoapi/datafusion/user_defined/index.rst.txt  |   2 +-
 _sources/contributor-guide/ffi.rst.txt             |   2 +-
 _sources/contributor-guide/introduction.rst.txt    |   2 +-
 .../common-operations/expressions.rst.txt          |   9 +-
 .../user-guide/common-operations/windows.rst.txt   |  52 +-
 _sources/user-guide/configuration.rst.txt          | 137 +++-
 _sources/user-guide/data-sources.rst.txt           |  43 +-
 _sources/user-guide/dataframe/index.rst.txt        |  50 ++
 autoapi/datafusion/context/index.html              |  36 +-
 autoapi/datafusion/dataframe/index.html            | 788 ++++++---------------
 autoapi/datafusion/expr/index.html                 | 198 ++++--
 autoapi/datafusion/functions/index.html            | 143 +++-
 autoapi/datafusion/index.html                      | 726 +++++--------------
 autoapi/datafusion/user_defined/index.html         |   2 +-
 contributor-guide/ffi.html                         |   2 +-
 contributor-guide/introduction.html                |   2 +-
 genindex.html                                      | 122 ++--
 objects.inv                                        | Bin 8733 -> 8586 bytes
 searchindex.js                                     |   2 +-
 user-guide/common-operations/aggregations.html     |  26 +-
 user-guide/common-operations/basic-info.html       |  46 +-
 user-guide/common-operations/expressions.html      | 101 +--
 user-guide/common-operations/functions.html        |  20 +-
 user-guide/common-operations/windows.html          |  52 +-
 user-guide/configuration.html                      | 146 +++-
 user-guide/data-sources.html                       |  44 +-
 user-guide/dataframe/index.html                    |  45 ++
 user-guide/introduction.html                       |   2 +-
 33 files changed, 1797 insertions(+), 1831 deletions(-)

diff --git a/_sources/autoapi/datafusion/context/index.rst.txt b/_sources/autoapi/datafusion/context/index.rst.txt
index 672346d6..4725845e 100644
--- a/_sources/autoapi/datafusion/context/index.rst.txt
+++ b/_sources/autoapi/datafusion/context/index.rst.txt
@@ -431,6 +431,18 @@ Module Contents
 
 
 
+   .. py:method:: _convert_file_sort_order(file_sort_order: collections.abc.Sequence[collections.abc.Sequence[datafusion.expr.SortKey]] | None) -> list[list[datafusion._internal.expr.SortExpr]] | None
+      :staticmethod:
+
+
+      Convert nested ``SortKey`` sequences into raw sort expressions.
+
+      Each ``SortKey`` can be a column name string, an ``Expr``, or a
+      ``SortExpr`` and will be converted using
+      :func:`datafusion.expr.sort_list_to_raw_sort_list`.
+
+
+
    .. py:method:: _convert_table_partition_cols(table_partition_cols: list[tuple[str, str | pyarrow.DataType]]) -> list[tuple[str, pyarrow.DataType]]
       :staticmethod:
 
@@ -627,7 +639,7 @@ Module Contents
 
 
 
-   .. py:method:: read_parquet(path: str | pathlib.Path, table_partition_cols: list[tuple[str, str | pyarrow.DataType]] | None = None, parquet_pruning: bool = True, file_extension: str = '.parquet', skip_metadata: bool = True, schema: pyarrow.Schema | None = None, file_sort_order: list[list[datafusion.expr.Expr | datafusion.expr.SortExpr]] | None = None) -> datafusion.dataframe.DataFrame
+   .. py:method:: read_parquet(path: str | pathlib.Path, table_partition_cols: list[tuple[str, str | pyarrow.DataType]] | None = None, parquet_pruning: bool = True, file_extension: str = '.parquet', skip_metadata: bool = True, schema: pyarrow.Schema | None = None, file_sort_order: collections.abc.Sequence[collections.abc.Sequence[datafusion.expr.SortKey]] | None = None) -> datafusion.dataframe.DataFrame
 
       Read a Parquet source into a :py:class:`~datafusion.dataframe.Dataframe`.
 
@@ -643,7 +655,9 @@ Module Contents
       :param schema: An optional schema representing the parquet files. If None,
                      the parquet reader will try to infer it based on data in the
                      file.
-      :param file_sort_order: Sort order for the file.
+      :param file_sort_order: Sort order for the file. Each sort key can be
+                              specified as a column name (``str``), an expression
+                              (``Expr``), or a ``SortExpr``.
 
       :returns: DataFrame representation of the read Parquet files
 
@@ -730,7 +744,7 @@ Module Contents
 
 
 
-   .. py:method:: register_listing_table(name: str, path: str | pathlib.Path, table_partition_cols: list[tuple[str, str | pyarrow.DataType]] | None = None, file_extension: str = '.parquet', schema: pyarrow.Schema | None = None, file_sort_order: list[list[datafusion.expr.Expr | datafusion.expr.SortExpr]] | None = None) -> None
+   .. py:method:: register_listing_table(name: str, path: str | pathlib.Path, table_partition_cols: list[tuple[str, str | pyarrow.DataType]] | None = None, file_extension: str = '.parquet', schema: pyarrow.Schema | None = None, file_sort_order: collections.abc.Sequence[collections.abc.Sequence[datafusion.expr.SortKey]] | None = None) -> None
 
       Register multiple files as a single table.
 
@@ -743,7 +757,9 @@ Module Contents
       :param table_partition_cols: Partition columns.
       :param file_extension: File extension of the provided table.
       :param schema: The data source schema.
-      :param file_sort_order: Sort order for the file.
+      :param file_sort_order: Sort order for the file. Each sort key can be
+                              specified as a column name (``str``), an expression
+                              (``Expr``), or a ``SortExpr``.
 
 
 
@@ -757,7 +773,7 @@ Module Contents
 
 
 
-   .. py:method:: register_parquet(name: str, path: str | pathlib.Path, table_partition_cols: list[tuple[str, str | pyarrow.DataType]] | None = None, parquet_pruning: bool = True, file_extension: str = '.parquet', skip_metadata: bool = True, schema: pyarrow.Schema | None = None, file_sort_order: list[list[datafusion.expr.SortExpr]] | None = None) -> None
+   .. py:method:: register_parquet(name: str, path: str | pathlib.Path, table_partition_cols: list[tuple[str, str | pyarrow.DataType]] | None = None, parquet_pruning: bool = True, file_extension: str = '.parquet', skip_metadata: bool = True, schema: pyarrow.Schema | None = None, file_sort_order: collections.abc.Sequence[collections.abc.Sequence[datafusion.expr.SortKey]] | None = None) -> None
 
       Register a Parquet file as a table.
 
@@ -775,7 +791,9 @@ Module Contents
                             that may be in the file schema. This can help avoid schema
                             conflicts due to metadata.
       :param schema: The data source schema.
-      :param file_sort_order: Sort order for the file.
+      :param file_sort_order: Sort order for the file. Each sort key can be
+                              specified as a column name (``str``), an expression
+                              (``Expr``), or a ``SortExpr``.
 
 
 
diff --git a/_sources/autoapi/datafusion/dataframe/index.rst.txt b/_sources/autoapi/datafusion/dataframe/index.rst.txt
index f0272cc7..cfc715bb 100644
--- a/_sources/autoapi/datafusion/dataframe/index.rst.txt
+++ b/_sources/autoapi/datafusion/dataframe/index.rst.txt
@@ -138,12 +138,12 @@ Module Contents
    .. py:method:: _repr_html_() -> str
 
 
-   .. py:method:: aggregate(group_by: list[datafusion.expr.Expr] | datafusion.expr.Expr, aggs: list[datafusion.expr.Expr] | datafusion.expr.Expr) -> DataFrame
+   .. py:method:: aggregate(group_by: collections.abc.Sequence[datafusion.expr.Expr | str] | datafusion.expr.Expr | str, aggs: collections.abc.Sequence[datafusion.expr.Expr] | datafusion.expr.Expr) -> DataFrame
 
       Aggregates the rows of the current DataFrame.
 
-      :param group_by: List of expressions to group by.
-      :param aggs: List of expressions to aggregate.
+      :param group_by: Sequence of expressions or column names to group by.
+      :param aggs: Sequence of expressions to aggregate.
 
       :returns: DataFrame after aggregation.
 
@@ -171,7 +171,7 @@ Module Contents
 
       Execute this :py:class:`DataFrame` and collect results into memory.
 
-      Prior to calling ``collect``, modifying a DataFrme simply updates a plan
+      Prior to calling ``collect``, modifying a DataFrame simply updates a plan
       (no actual computation is performed). Calling ``collect`` triggers the
       computation.
 
@@ -289,7 +289,7 @@ Module Contents
       If ``analyze`` is specified, runs the plan and reports metrics.
 
       :param verbose: If ``True``, more details will be included.
-      :param analyze: If ``Tru`e``, the plan will run and metrics reported.
+      :param analyze: If ``True``, the plan will run and metrics reported.
 
 
 
@@ -321,9 +321,17 @@ Module Contents
       Return a DataFrame for which ``predicate`` evaluates to ``True``.
 
       Rows for which ``predicate`` evaluates to ``False`` or ``None`` are filtered
-      out.  If more than one predicate is provided, these predicates will be
-      combined as a logical AND. If more complex logic is required, see the
-      logical operations in :py:mod:`~datafusion.functions`.
+      out. If more than one predicate is provided, these predicates will be
+      combined as a logical AND. Each ``predicate`` must be an
+      :class:`~datafusion.expr.Expr` created using helper functions such as
+      :func:`datafusion.col` or :func:`datafusion.lit`.
+      If more complex logic is required, see the logical operations in
+      :py:mod:`~datafusion.functions`.
+
+      Example::
+
+          from datafusion import col, lit
+          df.filter(col("a") > lit(1))
 
       :param predicates: Predicate expression(s) to filter the DataFrame.
 
@@ -359,8 +367,8 @@ Module Contents
 
 
 
-   .. py:method:: join(right: DataFrame, on: str | Sequence[str], how: Literal['inner', 'left', 'right', 'full', 'semi', 'anti'] = 'inner', *, left_on: None = None, right_on: None = None, join_keys: None = None) -> DataFrame
-                  join(right: DataFrame, on: None = None, how: Literal['inner', 'left', 'right', 'full', 'semi', 'anti'] = 'inner', *, left_on: str | Sequence[str], right_on: str | Sequence[str], join_keys: tuple[list[str], list[str]] | None = None) -> DataFrame
+   .. py:method:: join(right: DataFrame, on: str | collections.abc.Sequence[str], how: Literal['inner', 'left', 'right', 'full', 'semi', 'anti'] = 'inner', *, left_on: None = None, right_on: None = None, join_keys: None = None) -> DataFrame
+                  join(right: DataFrame, on: None = None, how: Literal['inner', 'left', 'right', 'full', 'semi', 'anti'] = 'inner', *, left_on: str | collections.abc.Sequence[str], right_on: str | collections.abc.Sequence[str], join_keys: tuple[list[str], list[str]] | None = None) -> DataFrame
                   join(right: DataFrame, on: None = None, how: Literal['inner', 'left', 'right', 'full', 'semi', 'anti'] = 'inner', *, join_keys: tuple[list[str], list[str]], left_on: None = None, right_on: None = None) -> DataFrame
 
       Join this :py:class:`DataFrame` with another :py:class:`DataFrame`.
@@ -383,8 +391,14 @@ Module Contents
 
       Join two :py:class:`DataFrame` using the specified expressions.
 
-      On expressions are used to support in-equality predicates. Equality
-      predicates are correctly optimized
+      Join predicates must be :class:`~datafusion.expr.Expr` objects, typically
+      built with :func:`datafusion.col`. On expressions are used to support
+      in-equality predicates. Equality predicates are correctly optimized.
+
+      Example::
+
+          from datafusion import col
+          df.join_on(other_df, col("id") == col("other_id"))
 
       :param right: Other DataFrame to join with.
       :param on_exprs: single or multiple (in)-equality predicates.
@@ -492,14 +506,14 @@ Module Contents
 
 
 
-   .. py:method:: sort(*exprs: datafusion.expr.Expr | datafusion.expr.SortExpr) -> DataFrame
+   .. py:method:: sort(*exprs: datafusion.expr.SortKey) -> DataFrame
 
-      Sort the DataFrame by the specified sorting expressions.
+      Sort the DataFrame by the specified sorting expressions or column names.
 
       Note that any expression can be turned into a sort expression by
-      calling its` ``sort`` method.
+      calling its ``sort`` method.
 
-      :param exprs: Sort expressions, applied in order.
+      :param exprs: Sort expressions or column names, applied in order.
 
       :returns: DataFrame after sorting.
 
@@ -622,6 +636,14 @@ Module Contents
 
       Add an additional column to the DataFrame.
 
+      The ``expr`` must be an :class:`~datafusion.expr.Expr` constructed with
+      :func:`datafusion.col` or :func:`datafusion.lit`.
+
+      Example::
+
+          from datafusion import col, lit
+          df.with_column("b", col("a") + lit(1))
+
       :param name: Name of the column to add.
       :param expr: Expression to compute the column.
 
@@ -649,16 +671,20 @@ Module Contents
 
       Add columns to the DataFrame.
 
-      By passing expressions, iteratables of expressions, or named expressions. To
-      pass named expressions use the form name=Expr.
+      By passing expressions, iterables of expressions, or named expressions.
+      All expressions must be :class:`~datafusion.expr.Expr` objects created via
+      :func:`datafusion.col` or :func:`datafusion.lit`.
+      To pass named expressions use the form ``name=Expr``.
 
-      Example usage: The following will add 4 columns labeled a, b, c, and d::
+      Example usage: The following will add 4 columns labeled ``a``, ``b``, ``c``,
+      and ``d``::
 
+          from datafusion import col, lit
           df = df.with_columns(
-              lit(0).alias('a'),
-              [lit(1).alias('b'), lit(2).alias('c')],
+              col("x").alias("a"),
+              [lit(1).alias("b"), col("y").alias("c")],
               d=lit(3)
-              )
+          )
 
       :param exprs: Either a single expression or an iterable of expressions to add.
       :param named_exprs: Named expressions in the form of ``name=expr``
@@ -726,51 +752,32 @@ Module Contents
    Parquet options for individual columns.
 
    Contains the available options that can be applied for an individual Parquet column,
-   replacing the global options in `ParquetWriterOptions`.
-
-   .. attribute:: encoding
-
-      Sets encoding for the column path. Valid values are: `plain`,
-      `plain_dictionary`, `rle`, `bit_packed`, `delta_binary_packed`,
-      `delta_length_byte_array`, `delta_byte_array`, `rle_dictionary`, and
-      `byte_stream_split`. These values are not case-sensitive. If `None`, uses
-      the default parquet options
-
-   .. attribute:: dictionary_enabled
-
-      Sets if dictionary encoding is enabled for the column path.
-      If `None`, uses the default parquet options
-
-   .. attribute:: compression
-
-      Sets default parquet compression codec for the column path. Valid
-      values are `uncompressed`, `snappy`, `gzip(level)`, `lzo`, `brotli(level)`,
-      `lz4`, `zstd(level)`, and `lz4_raw`. These values are not case-sensitive. If
-      `None`, uses the default parquet options.
-
-   .. attribute:: statistics_enabled
-
-      Sets if statistics are enabled for the column Valid values
-      are: `none`, `chunk`, and `page` These values are not case sensitive. If
-      `None`, uses the default parquet options.
-
-   .. attribute:: bloom_filter_enabled
-
-      Sets if bloom filter is enabled for the column path. If
-      `None`, uses the default parquet options.
-
-   .. attribute:: bloom_filter_fpp
-
-      Sets bloom filter false positive probability for the column
-      path. If `None`, uses the default parquet options.
-
-   .. attribute:: bloom_filter_ndv
-
-      Sets bloom filter number of distinct values. If `None`, uses
-      the default parquet options.
+   replacing the global options in ``ParquetWriterOptions``.
 
    Initialize the ParquetColumnOptions.
 
+   :param encoding: Sets encoding for the column path. Valid values are: ``plain``,
+                    ``plain_dictionary``, ``rle``, ``bit_packed``, ``delta_binary_packed``,
+                    ``delta_length_byte_array``, ``delta_byte_array``, ``rle_dictionary``,
+                    and ``byte_stream_split``. These values are not case-sensitive. If
+                    ``None``, uses the default parquet options
+   :param dictionary_enabled: Sets if dictionary encoding is enabled for the column
+                              path. If `None`, uses the default parquet options
+   :param compression: Sets default parquet compression codec for the column path.
+                       Valid values are ``uncompressed``, ``snappy``, ``gzip(level)``, ``lzo``,
+                       ``brotli(level)``, ``lz4``, ``zstd(level)``, and ``lz4_raw``. These
+                       values are not case-sensitive. If ``None``, uses the default parquet
+                       options.
+   :param statistics_enabled: Sets if statistics are enabled for the column Valid
+                              values are: ``none``, ``chunk``, and ``page`` These values are not case
+                              sensitive. If ``None``, uses the default parquet options.
+   :param bloom_filter_enabled: Sets if bloom filter is enabled for the column path.
+                                If ``None``, uses the default parquet options.
+   :param bloom_filter_fpp: Sets bloom filter false positive probability for the
+                            column path. If ``None``, uses the default parquet options.
+   :param bloom_filter_ndv: Sets bloom filter number of distinct values. If ``None``,
+                            uses the default parquet options.
+
 
    .. py:attribute:: bloom_filter_enabled
       :value: None
@@ -812,132 +819,74 @@ Module Contents
    Advanced parquet writer options.
 
    Allows settings the writer options that apply to the entire file. Some options can
-   also be set on a column by column basis, with the field `column_specific_options`
-   (see `ParquetColumnOptions`).
-
-   .. attribute:: data_pagesize_limit
-
-      Sets best effort maximum size of data page in bytes.
-
-   .. attribute:: write_batch_size
-
-      Sets write_batch_size in bytes.
-
-   .. attribute:: writer_version
-
-      Sets parquet writer version. Valid values are `1.0` and
-      `2.0`.
-
-   .. attribute:: skip_arrow_metadata
-
-      Skip encoding the embedded arrow metadata in the
-      KV_meta.
-
-   .. attribute:: compression
-
-      Compression type to use. Default is "zstd(3)".
-      Available compression types are
-      - "uncompressed": No compression.
-      - "snappy": Snappy compression.
-      - "gzip(n)": Gzip compression with level n.
-      - "brotli(n)": Brotli compression with level n.
-      - "lz4": LZ4 compression.
-      - "lz4_raw": LZ4_RAW compression.
-      - "zstd(n)": Zstandard compression with level n.
-
-   .. attribute:: dictionary_enabled
-
-      Sets if dictionary encoding is enabled. If None, uses
-      the default parquet writer setting.
-
-   .. attribute:: dictionary_page_size_limit
-
-      Sets best effort maximum dictionary page size,
-      in bytes.
-
-   .. attribute:: statistics_enabled
-
-      Sets if statistics are enabled for any column Valid
-      values are `none`, `chunk`, and `page`. If None, uses the default
-      parquet writer setting.
-
-   .. attribute:: max_row_group_size
-
-      Target maximum number of rows in each row group
-      (defaults to 1M rows). Writing larger row groups requires more memory to
-      write, but can get better compression and be faster to read.
-
-   .. attribute:: created_by
-
-      Sets "created by" property.
-
-   .. attribute:: column_index_truncate_length
-
-      Sets column index truncate length.
-
-   .. attribute:: statistics_truncate_length
-
-      Sets statistics truncate length. If None, uses
-      the default parquet writer setting.
-
-   .. attribute:: data_page_row_count_limit
-
-      Sets best effort maximum number of rows in a data
-      page.
-
-   .. attribute:: encoding
-
-      Sets default encoding for any column. Valid values are `plain`,
-      `plain_dictionary`, `rle`, `bit_packed`, `delta_binary_packed`,
-      `delta_length_byte_array`, `delta_byte_array`, `rle_dictionary`, and
-      `byte_stream_split`. If None, uses the default parquet writer setting.
-
-   .. attribute:: bloom_filter_on_write
-
-      Write bloom filters for all columns when creating
-      parquet files.
-
-   .. attribute:: bloom_filter_fpp
-
-      Sets bloom filter false positive probability. If None,
-      uses the default parquet writer setting
-
-   .. attribute:: bloom_filter_ndv
-
-      Sets bloom filter number of distinct values. If None, uses
-      the default parquet writer setting.
-
-   .. attribute:: allow_single_file_parallelism
-
-      Controls whether DataFusion will attempt to
-      speed up writing parquet files by serializing them in parallel. Each
-      column in each row group in each output file are serialized in parallel
-      leveraging a maximum possible core count of n_files * n_row_groups *
-      n_columns.
-
-   .. attribute:: maximum_parallel_row_group_writers
-
-      By default parallel parquet writer is
-      tuned for minimum memory usage in a streaming execution plan. You may
-      see a performance benefit when writing large parquet files by increasing
-      `maximum_parallel_row_group_writers` and
-      `maximum_buffered_record_batches_per_stream` if your system has idle
-      cores and can tolerate additional memory usage. Boosting these values is
-      likely worthwhile when writing out already in-memory data, such as from
-      a cached data frame.
-
-   .. attribute:: maximum_buffered_record_batches_per_stream
-
-      See
-      `maximum_parallel_row_group_writers`.
-
-   .. attribute:: column_specific_options
-
-      Overrides options for specific columns. If a column
-      is not a part of this dictionary, it will use the parameters provided here.
+   also be set on a column by column basis, with the field ``column_specific_options``
+   (see ``ParquetColumnOptions``).
 
    Initialize the ParquetWriterOptions.
 
+   :param data_pagesize_limit: Sets best effort maximum size of data page in bytes.
+   :param write_batch_size: Sets write_batch_size in bytes.
+   :param writer_version: Sets parquet writer version. Valid values are ``1.0`` and
+                          ``2.0``.
+   :param skip_arrow_metadata: Skip encoding the embedded arrow metadata in the
+                               KV_meta.
+   :param compression: Compression type to use. Default is ``zstd(3)``.
+                       Available compression types are
+
+                       - ``uncompressed``: No compression.
+                       - ``snappy``: Snappy compression.
+                       - ``gzip(n)``: Gzip compression with level n.
+                       - ``brotli(n)``: Brotli compression with level n.
+                       - ``lz4``: LZ4 compression.
+                       - ``lz4_raw``: LZ4_RAW compression.
+                       - ``zstd(n)``: Zstandard compression with level n.
+   :param compression_level: Compression level to set.
+   :param dictionary_enabled: Sets if dictionary encoding is enabled. If ``None``,
+                              uses the default parquet writer setting.
+   :param dictionary_page_size_limit: Sets best effort maximum dictionary page size,
+                                      in bytes.
+   :param statistics_enabled: Sets if statistics are enabled for any column Valid
+                              values are ``none``, ``chunk``, and ``page``. If ``None``, uses the
+                              default parquet writer setting.
+   :param max_row_group_size: Target maximum number of rows in each row group
+                              (defaults to 1M rows). Writing larger row groups requires more memory
+                              to write, but can get better compression and be faster to read.
+   :param created_by: Sets "created by" property.
+   :param column_index_truncate_length: Sets column index truncate length.
+   :param statistics_truncate_length: Sets statistics truncate length. If ``None``,
+                                      uses the default parquet writer setting.
+   :param data_page_row_count_limit: Sets best effort maximum number of rows in a data
+                                     page.
+   :param encoding: Sets default encoding for any column. Valid values are ``plain``,
+                    ``plain_dictionary``, ``rle``, ``bit_packed``, ``delta_binary_packed``,
+                    ``delta_length_byte_array``, ``delta_byte_array``, ``rle_dictionary``,
+                    and ``byte_stream_split``. If ``None``, uses the default parquet writer
+                    setting.
+   :param bloom_filter_on_write: Write bloom filters for all columns when creating
+                                 parquet files.
+   :param bloom_filter_fpp: Sets bloom filter false positive probability. If ``None``,
+                            uses the default parquet writer setting
+   :param bloom_filter_ndv: Sets bloom filter number of distinct values. If ``None``,
+                            uses the default parquet writer setting.
+   :param allow_single_file_parallelism: Controls whether DataFusion will attempt to
+                                         speed up writing parquet files by serializing them in parallel. Each
+                                         column in each row group in each output file are serialized in parallel
+                                         leveraging a maximum possible core count of
+                                         ``n_files * n_row_groups * n_columns``.
+   :param maximum_parallel_row_group_writers: By default parallel parquet writer is
+                                              tuned for minimum memory usage in a streaming execution plan. You may
+                                              see a performance benefit when writing large parquet files by increasing
+                                              ``maximum_parallel_row_group_writers`` and
+                                              ``maximum_buffered_record_batches_per_stream`` if your system has idle
+                                              cores and can tolerate additional memory usage. Boosting these values is
+                                              likely worthwhile when writing out already in-memory data, such as from
+                                              a cached data frame.
+   :param maximum_buffered_record_batches_per_stream: See
+                                                      ``maximum_parallel_row_group_writers``.
+   :param column_specific_options: Overrides options for specific columns. If a column
+                                   is not a part of this dictionary, it will use the parameters provided
+                                   here.
+
 
    .. py:attribute:: allow_single_file_parallelism
       :value: True
diff --git a/_sources/autoapi/datafusion/expr/index.rst.txt b/_sources/autoapi/datafusion/expr/index.rst.txt
index c5a8d517..71f68f7e 100644
--- a/_sources/autoapi/datafusion/expr/index.rst.txt
+++ b/_sources/autoapi/datafusion/expr/index.rst.txt
@@ -42,6 +42,7 @@ Attributes
    datafusion.expr.DropFunction
    datafusion.expr.DropTable
    datafusion.expr.DropView
+   datafusion.expr.EXPR_TYPE_ERROR
    datafusion.expr.EmptyRelation
    datafusion.expr.Execute
    datafusion.expr.Exists
@@ -81,6 +82,7 @@ Attributes
    datafusion.expr.SetVariable
    datafusion.expr.SimilarTo
    datafusion.expr.Sort
+   datafusion.expr.SortKey
    datafusion.expr.Subquery
    datafusion.expr.SubqueryAlias
    datafusion.expr.TableScan
@@ -110,6 +112,15 @@ Classes
    datafusion.expr.WindowFrameBound
 
 
+Functions
+---------
+
+.. autoapisummary::
+
+   datafusion.expr.ensure_expr
+   datafusion.expr.ensure_expr_list
+
+
 Module Contents
 ---------------
 
@@ -122,7 +133,7 @@ Module Contents
        import datafusion.functions as f
        from datafusion import lit, col
        df.select(
-           f.case(col("column_a")
+           f.case(col("column_a"))
            .when(lit(1), lit("One"))
            .when(lit(2), lit("Two"))
            .otherwise(lit("Unknown"))
@@ -203,7 +214,12 @@ Module Contents
 
       If ``key`` is a string, returns the subfield of the struct.
       If ``key`` is an integer, retrieves the element in the array. Note that the
-      element index begins at ``0``, unlike `array_element` which begins at ``1``.
+      element index begins at ``0``, unlike
+      :py:func:`~datafusion.functions.array_element` which begins at ``1``.
+      If ``key`` is a slice, returns an array that contains a slice of the
+      original array. Similar to integer indexing, this follows Python convention
+      where the index begins at ``0`` unlike
+      :py:func:`~datafusion.functions.array_slice` which begins at ``1``.
 
 
 
@@ -446,7 +462,7 @@ Module Contents
 
 
 
-   .. py:method:: cast(to: pyarrow.DataType[Any] | type[float | int | str | bool]) -> Expr
+   .. py:method:: cast(to: Union[pyarrow.DataType[Any], type]) -> Expr
 
       Cast to a new data type.
 
@@ -556,13 +572,13 @@ Module Contents
 
 
 
-   .. py:method:: fill_nan(value: Any | Expr | None = None) -> Expr
+   .. py:method:: fill_nan(value: Optional[Union[Any, Expr]] = None) -> Expr
 
       Fill NaN values with a provided value.
 
 
 
-   .. py:method:: fill_null(value: Any | Expr | None = None) -> Expr
+   .. py:method:: fill_null(value: Optional[Union[Any, Expr]] = None) -> Expr
 
       Fill NULL values with a provided value.
 
@@ -740,7 +756,7 @@ Module Contents
 
 
 
-   .. py:method:: order_by(*exprs: Expr | SortExpr) -> ExprFuncBuilder
+   .. py:method:: order_by(*exprs: Union[Expr, SortExpr]) -> ExprFuncBuilder
 
       Set the ordering for a window or aggregate function.
 
@@ -1032,7 +1048,7 @@ Module Contents
    .. py:attribute:: raw_sort
 
 
-.. py:class:: Window(partition_by: Optional[list[Expr] | Expr] = None, window_frame: Optional[WindowFrame] = None, order_by: Optional[list[SortExpr | Expr] | Expr | SortExpr] = None, null_treatment: Optional[datafusion.common.NullTreatment] = None)
+.. py:class:: Window(partition_by: Optional[Union[list[Expr], Expr]] = None, window_frame: Optional[WindowFrame] = None, order_by: Optional[Union[list[Union[SortExpr, Expr, str]], Expr, SortExpr, str]] = None, null_treatment: Optional[datafusion.common.NullTreatment] = None)
 
    Define reusable window parameters.
 
@@ -1115,7 +1131,7 @@ Module Contents
    Constructs a window frame bound.
 
 
-   .. py:method:: get_offset() -> int | None
+   .. py:method:: get_offset() -> Optional[int]
 
       Returns the offset of the window frame.
 
@@ -1148,6 +1164,32 @@ Module Contents
    .. py:attribute:: frame_bound
 
 
+.. py:function:: ensure_expr(value: Union[Expr, Any]) -> datafusion._internal.expr.Expr
+
+   Return the internal expression from ``Expr`` or raise ``TypeError``.
+
+   This helper rejects plain strings and other non-:class:`Expr` values so
+   higher level APIs consistently require explicit :func:`~datafusion.col` or
+   :func:`~datafusion.lit` expressions.
+
+   :param value: Candidate expression or other object.
+
+   :returns: The internal expression representation.
+
+   :raises TypeError: If ``value`` is not an instance of :class:`Expr`.
+
+
+.. py:function:: ensure_expr_list(exprs: Iterable[Union[Expr, Iterable[Expr]]]) -> list[datafusion._internal.expr.Expr]
+
+   Flatten an iterable of expressions, validating each via ``ensure_expr``.
+
+   :param exprs: Possibly nested iterable containing expressions.
+
+   :returns: A flat list of raw expressions.
+
+   :raises TypeError: If any item is not an instance of :class:`Expr`.
+
+
 .. py:data:: Aggregate
 
 .. py:data:: AggregateFunction
@@ -1200,6 +1242,10 @@ Module Contents
 
 .. py:data:: DropView
 
+.. py:data:: EXPR_TYPE_ERROR
+   :value: 'Use col()/column() or lit()/literal() to construct expressions'
+
+
 .. py:data:: EmptyRelation
 
 .. py:data:: Execute
@@ -1278,6 +1324,8 @@ Module Contents
 
 .. py:data:: Sort
 
+.. py:data:: SortKey
+
 .. py:data:: Subquery
 
 .. py:data:: SubqueryAlias
diff --git a/_sources/autoapi/datafusion/functions/index.rst.txt b/_sources/autoapi/datafusion/functions/index.rst.txt
index 26a92da0..6a2a049a 100644
--- a/_sources/autoapi/datafusion/functions/index.rst.txt
+++ b/_sources/autoapi/datafusion/functions/index.rst.txt
@@ -325,7 +325,7 @@ Module Contents
    :param filter: If provided, only compute against rows for which the filter is True
 
 
-.. py:function:: approx_percentile_cont(expression: datafusion.expr.Expr, percentile: float, num_centroids: Optional[int] = None, filter: Optional[datafusion.expr.Expr] = None) -> datafusion.expr.Expr
+.. py:function:: approx_percentile_cont(sort_expression: datafusion.expr.Expr | datafusion.expr.SortExpr, percentile: float, num_centroids: Optional[int] = None, filter: Optional[datafusion.expr.Expr] = None) -> datafusion.expr.Expr
 
    Returns the value that is approximately at a given percentile of ``expr``.
 
@@ -336,19 +336,19 @@ Module Contents
    between two of the values.
 
    This function uses the [t-digest](https://arxiv.org/abs/1902.04023) algorithm to
-   compute the percentil. You can limit the number of bins used in this algorithm by
+   compute the percentile. You can limit the number of bins used in this algorithm by
    setting the ``num_centroids`` parameter.
 
    If using the builder functions described in ref:`_aggregation` this function ignores
    the options ``order_by``, ``null_treatment``, and ``distinct``.
 
-   :param expression: Values for which to find the approximate percentile
+   :param sort_expression: Values for which to find the approximate percentile
    :param percentile: This must be between 0.0 and 1.0, inclusive
    :param num_centroids: Max bin size for the t-digest algorithm
    :param filter: If provided, only compute against rows for which the filter is True
 
 
-.. py:function:: approx_percentile_cont_with_weight(expression: datafusion.expr.Expr, weight: datafusion.expr.Expr, percentile: float, filter: Optional[datafusion.expr.Expr] = None) -> datafusion.expr.Expr
+.. py:function:: approx_percentile_cont_with_weight(sort_expression: datafusion.expr.Expr | datafusion.expr.SortExpr, weight: datafusion.expr.Expr, percentile: float, num_centroids: Optional[int] = None, filter: Optional[datafusion.expr.Expr] = None) -> datafusion.expr.Expr
 
    Returns the value of the weighted approximate percentile.
 
@@ -358,9 +358,10 @@ Module Contents
    If using the builder functions described in ref:`_aggregation` this function ignores
    the options ``order_by``, ``null_treatment``, and ``distinct``.
 
-   :param expression: Values for which to find the approximate percentile
+   :param sort_expression: Values for which to find the approximate percentile
    :param weight: Relative weight for each of the values in ``expression``
    :param percentile: This must be between 0.0 and 1.0, inclusive
+   :param num_centroids: Max bin size for the t-digest algorithm
    :param filter: If provided, only compute against rows for which the filter is True
 
 
@@ -371,7 +372,7 @@ Module Contents
    This is an alias for :py:func:`make_array`.
 
 
-.. py:function:: array_agg(expression: datafusion.expr.Expr, distinct: bool = False, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: array_agg(expression: datafusion.expr.Expr, distinct: bool = False, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Aggregate values into an array.
 
@@ -385,7 +386,11 @@ Module Contents
    :param expression: Values to combine into an array
    :param distinct: If True, a single entry for each distinct value will be in the result
    :param filter: If provided, only compute against rows for which the filter is True
-   :param order_by: Order the resultant array values
+   :param order_by: Order the resultant array values. Accepts column names or expressions.
+
+   For example::
+
+       df.select(array_agg(col("a"), order_by="b"))
 
 
 .. py:function:: array_append(array: datafusion.expr.Expr, element: datafusion.expr.Expr) -> datafusion.expr.Expr
@@ -882,12 +887,12 @@ Module Contents
    :param filter: If provided, only compute against rows for which the filter is True
 
 
-.. py:function:: cume_dist(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: cume_dist(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Create a cumulative distribution window function.
 
    This window function is similar to :py:func:`rank` except that the returned values
-   are the ratio of the row number to the total numebr of rows. Here is an example of a
+   are the ratio of the row number to the total number of rows. Here is an example of a
    dataframe with a window ordered by descending ``points`` and the associated
    cumulative distribution::
 
@@ -901,7 +906,12 @@ Module Contents
        +--------+-----------+
 
    :param partition_by: Expressions to partition the window frame on.
-   :param order_by: Set ordering within the window frame.
+   :param order_by: Set ordering within the window frame. Accepts
+                    column names or expressions.
+
+   For example::
+
+       cume_dist(order_by="points")
 
 
 .. py:function:: current_date() -> datafusion.expr.Expr
@@ -953,7 +963,7 @@ Module Contents
    Converts the argument from radians to degrees.
 
 
-.. py:function:: dense_rank(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: dense_rank(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Create a dense_rank window function.
 
@@ -971,7 +981,12 @@ Module Contents
        +--------+------------+
 
    :param partition_by: Expressions to partition the window frame on.
-   :param order_by: Set ordering within the window frame.
+   :param order_by: Set ordering within the window frame. Accepts
+                    column names or expressions.
+
+   For example::
+
+       dense_rank(order_by="points")
 
 
 .. py:function:: digest(value: datafusion.expr.Expr, method: datafusion.expr.Expr) -> datafusion.expr.Expr
@@ -1024,7 +1039,7 @@ Module Contents
    The string list is a string composed of substrings separated by ``,`` characters.
 
 
-.. py:function:: first_value(expression: datafusion.expr.Expr, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None, null_treatment: datafusion.common.NullTreatment = NullTreatment.RESPECT_NULLS) -> datafusion.expr.Expr
+.. py:function:: first_value(expression: datafusion.expr.Expr, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None, null_treatment: datafusion.common.NullTreatment = NullTreatment.RESPECT_NULLS) -> datafusion.expr.Expr
 
    Returns the first value in a group of values.
 
@@ -1035,9 +1050,14 @@ Module Contents
 
    :param expression: Argument to perform bitwise calculation on
    :param filter: If provided, only compute against rows for which the filter is True
-   :param order_by: Set the ordering of the expression to evaluate
+   :param order_by: Set the ordering of the expression to evaluate. Accepts
+                    column names or expressions.
    :param null_treatment: Assign whether to respect or ignore null values.
 
+   For example::
+
+       df.select(first_value(col("a"), order_by="ts"))
+
 
 .. py:function:: flatten(array: datafusion.expr.Expr) -> datafusion.expr.Expr
 
@@ -1082,13 +1102,13 @@ Module Contents
    Returns true if a given number is +0.0 or -0.0 otherwise returns false.
 
 
-.. py:function:: lag(arg: datafusion.expr.Expr, shift_offset: int = 1, default_value: Optional[Any] = None, partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: lag(arg: datafusion.expr.Expr, shift_offset: int = 1, default_value: Optional[Any] = None, partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Create a lag window function.
 
    Lag operation will return the argument that is in the previous shift_offset-th row
    in the partition. For example ``lag(col("b"), shift_offset=3, default_value=5)``
-   will return the 3rd previous value in column ``b``. At the beginnig of the
+   will return the 3rd previous value in column ``b``. At the beginning of the
    partition, where no values can be returned it will return the default value of 5.
 
    Here is an example of both the ``lag`` and :py:func:`datafusion.functions.lead`
@@ -1107,10 +1127,15 @@ Module Contents
    :param shift_offset: Number of rows before the current row.
    :param default_value: Value to return if shift_offet row does not exist.
    :param partition_by: Expressions to partition the window frame on.
-   :param order_by: Set ordering within the window frame.
+   :param order_by: Set ordering within the window frame. Accepts
+                    column names or expressions.
+
+   For example::
 
+       lag(col("b"), order_by="ts")
 
-.. py:function:: last_value(expression: datafusion.expr.Expr, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None, null_treatment: datafusion.common.NullTreatment = NullTreatment.RESPECT_NULLS) -> datafusion.expr.Expr
+
+.. py:function:: last_value(expression: datafusion.expr.Expr, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None, null_treatment: datafusion.common.NullTreatment = NullTreatment.RESPECT_NULLS) -> datafusion.expr.Expr
 
    Returns the last value in a group of values.
 
@@ -1121,23 +1146,28 @@ Module Contents
 
    :param expression: Argument to perform bitwise calculation on
    :param filter: If provided, only compute against rows for which the filter is True
-   :param order_by: Set the ordering of the expression to evaluate
+   :param order_by: Set the ordering of the expression to evaluate. Accepts
+                    column names or expressions.
    :param null_treatment: Assign whether to respect or ignore null values.
 
+   For example::
+
+       df.select(last_value(col("a"), order_by="ts"))
+
 
 .. py:function:: lcm(x: datafusion.expr.Expr, y: datafusion.expr.Expr) -> datafusion.expr.Expr
 
    Returns the least common multiple.
 
 
-.. py:function:: lead(arg: datafusion.expr.Expr, shift_offset: int = 1, default_value: Optional[Any] = None, partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: lead(arg: datafusion.expr.Expr, shift_offset: int = 1, default_value: Optional[Any] = None, partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Create a lead window function.
 
    Lead operation will return the argument that is in the next shift_offset-th row in
    the partition. For example ``lead(col("b"), shift_offset=3, default_value=5)`` will
    return the 3rd following value in column ``b``. At the end of the partition, where
-   no futher values can be returned it will return the default value of 5.
+   no further values can be returned it will return the default value of 5.
 
    Here is an example of both the ``lead`` and :py:func:`datafusion.functions.lag`
    functions on a simple DataFrame::
@@ -1158,7 +1188,12 @@ Module Contents
    :param shift_offset: Number of rows following the current row.
    :param default_value: Value to return if shift_offet row does not exist.
    :param partition_by: Expressions to partition the window frame on.
-   :param order_by: Set ordering within the window frame.
+   :param order_by: Set ordering within the window frame. Accepts
+                    column names or expressions.
+
+   For example::
+
+       lead(col("b"), order_by="ts")
 
 
 .. py:function:: left(string: datafusion.expr.Expr, n: datafusion.expr.Expr) -> datafusion.expr.Expr
@@ -1512,7 +1547,7 @@ Module Contents
    This will use the same value for all instances of now() in same statement.
 
 
-.. py:function:: nth_value(expression: datafusion.expr.Expr, n: int, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None, null_treatment: datafusion.common.NullTreatment = NullTreatment.RESPECT_NULLS) -> datafusion.expr.Expr
+.. py:function:: nth_value(expression: datafusion.expr.Expr, n: int, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None, null_treatment: datafusion.common.NullTreatment = NullTreatment.RESPECT_NULLS) -> datafusion.expr.Expr
 
    Returns the n-th value in a group of values.
 
@@ -1524,11 +1559,16 @@ Module Contents
    :param expression: Argument to perform bitwise calculation on
    :param n: Index of value to return. Starts at 1.
    :param filter: If provided, only compute against rows for which the filter is True
-   :param order_by: Set the ordering of the expression to evaluate
+   :param order_by: Set the ordering of the expression to evaluate. Accepts
+                    column names or expressions.
    :param null_treatment: Assign whether to respect or ignore null values.
 
+   For example::
 
-.. py:function:: ntile(groups: int, partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+       df.select(nth_value(col("a"), 2, order_by="ts"))
+
+
+.. py:function:: ntile(groups: int, partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Create a n-tile window function.
 
@@ -1550,7 +1590,12 @@ Module Contents
 
    :param groups: Number of groups for the n-tile to be divided into.
    :param partition_by: Expressions to partition the window frame on.
-   :param order_by: Set ordering within the window frame.
+   :param order_by: Set ordering within the window frame. Accepts
+                    column names or expressions.
+
+   For example::
+
+       ntile(3, order_by="points")
 
 
 .. py:function:: nullif(expr1: datafusion.expr.Expr, expr2: datafusion.expr.Expr) -> datafusion.expr.Expr
@@ -1583,7 +1628,7 @@ Module Contents
    extends for ``length`` characters with new substring.
 
 
-.. py:function:: percent_rank(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: percent_rank(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Create a percent_rank window function.
 
@@ -1602,7 +1647,12 @@ Module Contents
        +--------+--------------+
 
    :param partition_by: Expressions to partition the window frame on.
-   :param order_by: Set ordering within the window frame.
+   :param order_by: Set ordering within the window frame. Accepts
+                    column names or expressions.
+
+   For example::
+
+       percent_rank(order_by="points")
 
 
 .. py:function:: pi() -> datafusion.expr.Expr
@@ -1637,13 +1687,13 @@ Module Contents
    Create a list of values in the range between start and stop.
 
 
-.. py:function:: rank(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: rank(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Create a rank window function.
 
    Returns the rank based upon the window order. Consecutive equal values will receive
    the same rank, but the next different value will not be consecutive but rather the
-   number of rows that preceed it plus one. This is similar to Olympic medals. If two
+   number of rows that precede it plus one. This is similar to Olympic medals. If two
    people tie for gold, the next place is bronze. There would be no silver medal. Here
    is an example of a dataframe with a window ordered by descending ``points`` and the
    associated rank.
@@ -1660,7 +1710,12 @@ Module Contents
        +--------+------+
 
    :param partition_by: Expressions to partition the window frame on.
-   :param order_by: Set ordering within the window frame.
+   :param order_by: Set ordering within the window frame. Accepts
+                    column names or expressions.
+
+   For example::
+
+       rank(order_by="points")
 
 
 .. py:function:: regexp_count(string: datafusion.expr.Expr, pattern: datafusion.expr.Expr, start: datafusion.expr.Expr, flags: datafusion.expr.Expr | None = None) -> datafusion.expr.Expr
@@ -1862,7 +1917,7 @@ Module Contents
    ``round(lit(125.2345), lit(-2))`` would yield a value of ``100.0``.
 
 
-.. py:function:: row_number(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: row_number(partition_by: Optional[list[datafusion.expr.Expr] | datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Create a row number window function.
 
@@ -1880,7 +1935,12 @@ Module Contents
        +--------+------------+
 
    :param partition_by: Expressions to partition the window frame on.
-   :param order_by: Set ordering within the window frame.
+   :param order_by: Set ordering within the window frame. Accepts
+                    column names or expressions.
+
+   For example::
+
+       row_number(order_by="points")
 
 
 .. py:function:: rpad(string: datafusion.expr.Expr, count: datafusion.expr.Expr, characters: datafusion.expr.Expr | None = None) -> datafusion.expr.Expr
@@ -1978,12 +2038,12 @@ Module Contents
    This is an alias for :py:func:`stddev`.
 
 
-.. py:function:: string_agg(expression: datafusion.expr.Expr, delimiter: str, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr] = None) -> datafusion.expr.Expr
+.. py:function:: string_agg(expression: datafusion.expr.Expr, delimiter: str, filter: Optional[datafusion.expr.Expr] = None, order_by: Optional[list[datafusion.expr.SortKey] | datafusion.expr.SortKey] = None) -> datafusion.expr.Expr
 
    Concatenates the input strings.
 
    This aggregate function will concatenate input strings, ignoring null values, and
-   seperating them with the specified delimiter. Non-string values will be converted to
+   separating them with the specified delimiter. Non-string values will be converted to
    their string equivalents.
 
    If using the builder functions described in ref:`_aggregation` this function ignores
@@ -1992,7 +2052,12 @@ Module Contents
    :param expression: Argument to perform bitwise calculation on
    :param delimiter: Text to place between each value of expression
    :param filter: If provided, only compute against rows for which the filter is True
-   :param order_by: Set the ordering of the expression to evaluate
+   :param order_by: Set the ordering of the expression to evaluate. Accepts
+                    column names or expressions.
+
+   For example::
+
+       df.select(string_agg(col("a"), ",", order_by="b"))
 
 
 .. py:function:: strpos(string: datafusion.expr.Expr, substring: datafusion.expr.Expr) -> datafusion.expr.Expr
@@ -2163,7 +2228,7 @@ Module Contents
    detailed usage.
 
 
-.. py:function:: window(name: str, args: list[datafusion.expr.Expr], partition_by: list[datafusion.expr.Expr] | datafusion.expr.Expr | None = None, order_by: list[datafusion.expr.Expr | datafusion.expr.SortExpr] | datafusion.expr.Expr | datafusion.expr.SortExpr | None = None, window_frame: datafusion.expr.WindowFrame | None = None, ctx: datafusion.context.SessionContext | None = None) -> datafusion.expr.Expr
+.. py:function:: window(name: str, args: list[datafusion.expr.Expr], partition_by: list[datafusion.expr.Expr] | datafusion.expr.Expr | None = None, order_by: list[datafusion.expr.SortKey] | datafusion.expr.SortKey | None = None, window_frame: datafusion.expr.WindowFrame | None = None, filter: datafusion.expr.Expr | None = None, distinct: bool = False, ctx: datafusion.context.SessionContext | None = None) -> datafusion.expr.Expr
 
    Creates a new Window function expression.
 
@@ -2173,4 +2238,8 @@ Module Contents
 
        df.select(functions.lag(col("a")).partition_by(col("b")).build())
 
+   The ``order_by`` parameter accepts column names or expressions, e.g.::
+
+       window("lag", [col("a")], order_by="ts")
+
 
diff --git a/_sources/autoapi/datafusion/index.rst.txt b/_sources/autoapi/datafusion/index.rst.txt
index 0b19b1d4..507c6ee1 100644
--- a/_sources/autoapi/datafusion/index.rst.txt
+++ b/_sources/autoapi/datafusion/index.rst.txt
@@ -424,7 +424,12 @@ Package Contents
 
       If ``key`` is a string, returns the subfield of the struct.
       If ``key`` is an integer, retrieves the element in the array. Note that the
-      element index begins at ``0``, unlike `array_element` which begins at ``1``.
+      element index begins at ``0``, unlike
+      :py:func:`~datafusion.functions.array_element` which begins at ``1``.
+      If ``key`` is a slice, returns an array that contains a slice of the
+      original array. Similar to integer indexing, this follows Python convention
+      where the index begins at ``0`` unlike
+      :py:func:`~datafusion.functions.array_slice` which begins at ``1``.
 
 
 
@@ -667,7 +672,7 @@ Package Contents
 
 
 
-   .. py:method:: cast(to: pyarrow.DataType[Any] | type[float | int | str | bool]) -> Expr
+   .. py:method:: cast(to: Union[pyarrow.DataType[Any], type]) -> Expr
 
       Cast to a new data type.
 
@@ -777,13 +782,13 @@ Package Contents
 
 
 
-   .. py:method:: fill_nan(value: Any | Expr | None = None) -> Expr
+   .. py:method:: fill_nan(value: Optional[Union[Any, Expr]] = None) -> Expr
 
       Fill NaN values with a provided value.
 
 
 
-   .. py:method:: fill_null(value: Any | Expr | None = None) -> Expr
+   .. py:method:: fill_null(value: Optional[Union[Any, Expr]] = None) -> Expr
 
       Fill NULL values with a provided value.
 
@@ -961,7 +966,7 @@ Package Contents
 
 
 
-   .. py:method:: order_by(*exprs: Expr | SortExpr) -> ExprFuncBuilder
+   .. py:method:: order_by(*exprs: Union[Expr, SortExpr]) -> ExprFuncBuilder
 
       Set the ordering for a window or aggregate function.
 
@@ -1308,51 +1313,32 @@ Package Contents
    Parquet options for individual columns.
 
    Contains the available options that can be applied for an individual Parquet column,
-   replacing the global options in `ParquetWriterOptions`.
-
-   .. attribute:: encoding
-
-      Sets encoding for the column path. Valid values are: `plain`,
-      `plain_dictionary`, `rle`, `bit_packed`, `delta_binary_packed`,
-      `delta_length_byte_array`, `delta_byte_array`, `rle_dictionary`, and
-      `byte_stream_split`. These values are not case-sensitive. If `None`, uses
-      the default parquet options
-
-   .. attribute:: dictionary_enabled
-
-      Sets if dictionary encoding is enabled for the column path.
-      If `None`, uses the default parquet options
-
-   .. attribute:: compression
-
-      Sets default parquet compression codec for the column path. Valid
-      values are `uncompressed`, `snappy`, `gzip(level)`, `lzo`, `brotli(level)`,
-      `lz4`, `zstd(level)`, and `lz4_raw`. These values are not case-sensitive. If
-      `None`, uses the default parquet options.
-
-   .. attribute:: statistics_enabled
-
-      Sets if statistics are enabled for the column Valid values
-      are: `none`, `chunk`, and `page` These values are not case sensitive. If
-      `None`, uses the default parquet options.
-
-   .. attribute:: bloom_filter_enabled
-
-      Sets if bloom filter is enabled for the column path. If
-      `None`, uses the default parquet options.
-
-   .. attribute:: bloom_filter_fpp
-
-      Sets bloom filter false positive probability for the column
-      path. If `None`, uses the default parquet options.
-
-   .. attribute:: bloom_filter_ndv
-
-      Sets bloom filter number of distinct values. If `None`, uses
-      the default parquet options.
+   replacing the global options in ``ParquetWriterOptions``.
 
    Initialize the ParquetColumnOptions.
 
+   :param encoding: Sets encoding for the column path. Valid values are: ``plain``,
+                    ``plain_dictionary``, ``rle``, ``bit_packed``, ``delta_binary_packed``,
+                    ``delta_length_byte_array``, ``delta_byte_array``, ``rle_dictionary``,
+                    and ``byte_stream_split``. These values are not case-sensitive. If
+                    ``None``, uses the default parquet options
+   :param dictionary_enabled: Sets if dictionary encoding is enabled for the column
+                              path. If `None`, uses the default parquet options
+   :param compression: Sets default parquet compression codec for the column path.
+                       Valid values are ``uncompressed``, ``snappy``, ``gzip(level)``, ``lzo``,
+                       ``brotli(level)``, ``lz4``, ``zstd(level)``, and ``lz4_raw``. These
+                       values are not case-sensitive. If ``None``, uses the default parquet
+                       options.
+   :param statistics_enabled: Sets if statistics are enabled for the column Valid
+                              values are: ``none``, ``chunk``, and ``page`` These values are not case
+                              sensitive. If ``None``, uses the default parquet options.
+   :param bloom_filter_enabled: Sets if bloom filter is enabled for the column path.
+                                If ``None``, uses the default parquet options.
+   :param bloom_filter_fpp: Sets bloom filter false positive probability for the
+                            column path. If ``None``, uses the default parquet options.
+   :param bloom_filter_ndv: Sets bloom filter number of distinct values. If ``None``,
+                            uses the default parquet options.
+
 
    .. py:attribute:: bloom_filter_enabled
       :value: None
@@ -1394,132 +1380,74 @@ Package Contents
    Advanced parquet writer options.
 
    Allows settings the writer options that apply to the entire file. Some options can
-   also be set on a column by column basis, with the field `column_specific_options`
-   (see `ParquetColumnOptions`).
-
-   .. attribute:: data_pagesize_limit
-
-      Sets best effort maximum size of data page in bytes.
-
-   .. attribute:: write_batch_size
-
-      Sets write_batch_size in bytes.
-
-   .. attribute:: writer_version
-
-      Sets parquet writer version. Valid values are `1.0` and
-      `2.0`.
-
-   .. attribute:: skip_arrow_metadata
-
-      Skip encoding the embedded arrow metadata in the
-      KV_meta.
-
-   .. attribute:: compression
-
-      Compression type to use. Default is "zstd(3)".
-      Available compression types are
-      - "uncompressed": No compression.
-      - "snappy": Snappy compression.
-      - "gzip(n)": Gzip compression with level n.
-      - "brotli(n)": Brotli compression with level n.
-      - "lz4": LZ4 compression.
-      - "lz4_raw": LZ4_RAW compression.
-      - "zstd(n)": Zstandard compression with level n.
-
-   .. attribute:: dictionary_enabled
-
-      Sets if dictionary encoding is enabled. If None, uses
-      the default parquet writer setting.
-
-   .. attribute:: dictionary_page_size_limit
-
-      Sets best effort maximum dictionary page size,
-      in bytes.
-
-   .. attribute:: statistics_enabled
-
-      Sets if statistics are enabled for any column Valid
-      values are `none`, `chunk`, and `page`. If None, uses the default
-      parquet writer setting.
-
-   .. attribute:: max_row_group_size
-
-      Target maximum number of rows in each row group
-      (defaults to 1M rows). Writing larger row groups requires more memory to
-      write, but can get better compression and be faster to read.
-
-   .. attribute:: created_by
-
-      Sets "created by" property.
-
-   .. attribute:: column_index_truncate_length
-
-      Sets column index truncate length.
-
-   .. attribute:: statistics_truncate_length
-
-      Sets statistics truncate length. If None, uses
-      the default parquet writer setting.
-
-   .. attribute:: data_page_row_count_limit
-
-      Sets best effort maximum number of rows in a data
-      page.
-
-   .. attribute:: encoding
-
-      Sets default encoding for any column. Valid values are `plain`,
-      `plain_dictionary`, `rle`, `bit_packed`, `delta_binary_packed`,
-      `delta_length_byte_array`, `delta_byte_array`, `rle_dictionary`, and
-      `byte_stream_split`. If None, uses the default parquet writer setting.
-
-   .. attribute:: bloom_filter_on_write
-
-      Write bloom filters for all columns when creating
-      parquet files.
-
-   .. attribute:: bloom_filter_fpp
-
-      Sets bloom filter false positive probability. If None,
-      uses the default parquet writer setting
-
-   .. attribute:: bloom_filter_ndv
-
-      Sets bloom filter number of distinct values. If None, uses
-      the default parquet writer setting.
-
-   .. attribute:: allow_single_file_parallelism
-
-      Controls whether DataFusion will attempt to
-      speed up writing parquet files by serializing them in parallel. Each
-      column in each row group in each output file are serialized in parallel
-      leveraging a maximum possible core count of n_files * n_row_groups *
-      n_columns.
-
-   .. attribute:: maximum_parallel_row_group_writers
-
-      By default parallel parquet writer is
-      tuned for minimum memory usage in a streaming execution plan. You may
-      see a performance benefit when writing large parquet files by increasing
-      `maximum_parallel_row_group_writers` and
-      `maximum_buffered_record_batches_per_stream` if your system has idle
-      cores and can tolerate additional memory usage. Boosting these values is
-      likely worthwhile when writing out already in-memory data, such as from
-      a cached data frame.
-
-   .. attribute:: maximum_buffered_record_batches_per_stream
-
-      See
-      `maximum_parallel_row_group_writers`.
-
-   .. attribute:: column_specific_options
-
-      Overrides options for specific columns. If a column
-      is not a part of this dictionary, it will use the parameters provided here.
+   also be set on a column by column basis, with the field ``column_specific_options``
+   (see ``ParquetColumnOptions``).
 
    Initialize the ParquetWriterOptions.
 
+   :param data_pagesize_limit: Sets best effort maximum size of data page in bytes.
+   :param write_batch_size: Sets write_batch_size in bytes.
+   :param writer_version: Sets parquet writer version. Valid values are ``1.0`` and
+                          ``2.0``.
+   :param skip_arrow_metadata: Skip encoding the embedded arrow metadata in the
+                               KV_meta.
+   :param compression: Compression type to use. Default is ``zstd(3)``.
+                       Available compression types are
+
+                       - ``uncompressed``: No compression.
+                       - ``snappy``: Snappy compression.
+                       - ``gzip(n)``: Gzip compression with level n.
+                       - ``brotli(n)``: Brotli compression with level n.
+                       - ``lz4``: LZ4 compression.
+                       - ``lz4_raw``: LZ4_RAW compression.
+                       - ``zstd(n)``: Zstandard compression with level n.
+   :param compression_level: Compression level to set.
+   :param dictionary_enabled: Sets if dictionary encoding is enabled. If ``None``,
+                              uses the default parquet writer setting.
+   :param dictionary_page_size_limit: Sets best effort maximum dictionary page size,
+                                      in bytes.
+   :param statistics_enabled: Sets if statistics are enabled for any column Valid
+                              values are ``none``, ``chunk``, and ``page``. If ``None``, uses the
+                              default parquet writer setting.
+   :param max_row_group_size: Target maximum number of rows in each row group
+                              (defaults to 1M rows). Writing larger row groups requires more memory
+                              to write, but can get better compression and be faster to read.
+   :param created_by: Sets "created by" property.
+   :param column_index_truncate_length: Sets column index truncate length.
+   :param statistics_truncate_length: Sets statistics truncate length. If ``None``,
+                                      uses the default parquet writer setting.
+   :param data_page_row_count_limit: Sets best effort maximum number of rows in a data
+                                     page.
+   :param encoding: Sets default encoding for any column. Valid values are ``plain``,
+                    ``plain_dictionary``, ``rle``, ``bit_packed``, ``delta_binary_packed``,
+                    ``delta_length_byte_array``, ``delta_byte_array``, ``rle_dictionary``,
+                    and ``byte_stream_split``. If ``None``, uses the default parquet writer
+                    setting.
+   :param bloom_filter_on_write: Write bloom filters for all columns when creating
+                                 parquet files.
+   :param bloom_filter_fpp: Sets bloom filter false positive probability. If ``None``,
+                            uses the default parquet writer setting
+   :param bloom_filter_ndv: Sets bloom filter number of distinct values. If ``None``,
+                            uses the default parquet writer setting.
+   :param allow_single_file_parallelism: Controls whether DataFusion will attempt to
+                                         speed up writing parquet files by serializing them in parallel. Each
+                                         column in each row group in each output file are serialized in parallel
+                                         leveraging a maximum possible core count of
+                                         ``n_files * n_row_groups * n_columns``.
+   :param maximum_parallel_row_group_writers: By default parallel parquet writer is
+                                              tuned for minimum memory usage in a streaming execution plan. You may
+                                              see a performance benefit when writing large parquet files by increasing
+                                              ``maximum_parallel_row_group_writers`` and
+                                              ``maximum_buffered_record_batches_per_stream`` if your system has idle
+                                              cores and can tolerate additional memory usage. Boosting these values is
+                                              likely worthwhile when writing out already in-memory data, such as from
+                                              a cached data frame.
+   :param maximum_buffered_record_batches_per_stream: See
+                                                      ``maximum_parallel_row_group_writers``.
+   :param column_specific_options: Overrides options for specific columns. If a column
+                                   is not a part of this dictionary, it will use the parameters provided
+                                   here.
+
 
    .. py:attribute:: allow_single_file_parallelism
       :value: True
diff --git a/_sources/autoapi/datafusion/user_defined/index.rst.txt b/_sources/autoapi/datafusion/user_defined/index.rst.txt
index 728ace0a..591d4cda 100644
--- a/_sources/autoapi/datafusion/user_defined/index.rst.txt
+++ b/_sources/autoapi/datafusion/user_defined/index.rst.txt
@@ -535,7 +535,7 @@ Module Contents
 
    .. py:method:: get_range(idx: int, num_rows: int) -> tuple[int, int]
 
-      Return the range for the window fuction.
+      Return the range for the window function.
 
       If `uses_window_frame` flag is `false`. This method is used to
       calculate required range for the window function during
diff --git a/_sources/contributor-guide/ffi.rst.txt b/_sources/contributor-guide/ffi.rst.txt
index a40af123..e201db71 100644
--- a/_sources/contributor-guide/ffi.rst.txt
+++ b/_sources/contributor-guide/ffi.rst.txt
@@ -195,7 +195,7 @@ optimization levels. If you wish to go down this route, there are two approaches
 have identified you can use.
 
 #. Re-export all of ``datafusion-python`` yourself with your extensions built in.
-#. Carefully synchonize your software releases with the ``datafusion-python`` CI build
+#. Carefully synchronize your software releases with the ``datafusion-python`` CI build
    system so that your libraries use the exact same compiler, features, and
    optimization level.
 
diff --git a/_sources/contributor-guide/introduction.rst.txt b/_sources/contributor-guide/introduction.rst.txt
index 2fba6411..6cb05c62 100644
--- a/_sources/contributor-guide/introduction.rst.txt
+++ b/_sources/contributor-guide/introduction.rst.txt
@@ -43,7 +43,7 @@ Bootstrap:
 
     # fetch this repo
     git clone git@github.com:apache/datafusion-python.git
-    # create the virtual enviornment
+    # create the virtual environment
     uv sync --dev --no-install-package datafusion
     # activate the environment
     source .venv/bin/activate
diff --git a/_sources/user-guide/common-operations/expressions.rst.txt b/_sources/user-guide/common-operations/expressions.rst.txt
index e94e1a6b..7848b4ee 100644
--- a/_sources/user-guide/common-operations/expressions.rst.txt
+++ b/_sources/user-guide/common-operations/expressions.rst.txt
@@ -64,7 +64,7 @@ Arrays
 ------
 
 For columns that contain arrays of values, you can access individual elements of the array by index
-using bracket indexing. This is similar to callling the function
+using bracket indexing. This is similar to calling the function
 :py:func:`datafusion.functions.array_element`, except that array indexing using brackets is 0 based,
 similar to Python arrays and ``array_element`` is 1 based indexing to be compatible with other SQL
 approaches.
@@ -82,6 +82,13 @@ approaches.
     Indexing an element of an array via ``[]`` starts at index 0 whereas
     :py:func:`~datafusion.functions.array_element` starts at index 1.
 
+Starting in DataFusion 49.0.0 you can also create slices of array elements using
+slice syntax from Python.
+
+.. ipython:: python
+
+    df.select(col("a")[1:3].alias("second_two_elements"))
+
 To check if an array is empty, you can use the function :py:func:`datafusion.functions.array_empty` or `datafusion.functions.empty`.
 This function returns a boolean indicating whether the array is empty.
 
diff --git a/_sources/user-guide/common-operations/windows.rst.txt b/_sources/user-guide/common-operations/windows.rst.txt
index 8225d125..c8fdea8f 100644
--- a/_sources/user-guide/common-operations/windows.rst.txt
+++ b/_sources/user-guide/common-operations/windows.rst.txt
@@ -24,14 +24,14 @@ In this section you will learn about window functions. A window function utilize
 multiple rows to produce a result for each individual row, unlike an aggregate function that
 provides a single value for multiple rows.
 
-The window functions are availble in the :py:mod:`~datafusion.functions` module.
+The window functions are available in the :py:mod:`~datafusion.functions` module.
 
 We'll use the pokemon dataset (from Ritchie Vink) in the following examples.
 
 .. ipython:: python
 
     from datafusion import SessionContext
-    from datafusion import col
+    from datafusion import col, lit
     from datafusion import functions as f
 
     ctx = SessionContext()
@@ -99,8 +99,8 @@ If you do not specify a Window Frame, the frame will be set depending on the fol
 criteria.
 
 * If an ``order_by`` clause is set, the default window frame is defined as the rows between
-  unbounded preceeding and the current row.
-* If an ``order_by`` is not set, the default frame is defined as the rows betwene unbounded
+  unbounded preceding and the current row.
+* If an ``order_by`` is not set, the default frame is defined as the rows between unbounded
   and unbounded following (the entire partition).
 
 Window Frames are defined by three parameters: unit type, starting bound, and ending bound.
@@ -116,20 +116,18 @@ The unit types available are:
   ``order_by`` clause.
 
 In this example we perform a "rolling average" of the speed of the current Pokemon and the
-two preceeding rows.
+two preceding rows.
 
 .. ipython:: python
 
-    from datafusion.expr import WindowFrame
+    from datafusion.expr import Window, WindowFrame
 
     df.select(
         col('"Name"'),
         col('"Speed"'),
-        f.window("avg",
-            [col('"Speed"')],
-            order_by=[col('"Speed"')],
-            window_frame=WindowFrame("rows", 2, 0)
-        ).alias("Previous Speed")
+        f.avg(col('"Speed"'))
+        .over(Window(window_frame=WindowFrame("rows", 2, 0), order_by=[col('"Speed"')]))
+        .alias("Previous Speed"),
     )
 
 Null Treatment
@@ -151,21 +149,27 @@ it's ``Type 2`` column that are null.
 
     from datafusion.common import NullTreatment
 
-    df.filter(col('"Type 1"') ==  lit("Bug")).select(
+    df.filter(col('"Type 1"') == lit("Bug")).select(
         '"Name"',
         '"Type 2"',
-        f.window("last_value", [col('"Type 2"')])
-            .window_frame(WindowFrame("rows", None, 0))
-            .order_by(col('"Speed"'))
-            .null_treatment(NullTreatment.IGNORE_NULLS)
-            .build()
-            .alias("last_wo_null"),
-        f.window("last_value", [col('"Type 2"')])
-            .window_frame(WindowFrame("rows", None, 0))
-            .order_by(col('"Speed"'))
-            .null_treatment(NullTreatment.RESPECT_NULLS)
-            .build()
-            .alias("last_with_null")
+        f.last_value(col('"Type 2"'))
+        .over(
+            Window(
+                window_frame=WindowFrame("rows", None, 0),
+                order_by=[col('"Speed"')],
+                null_treatment=NullTreatment.IGNORE_NULLS,
+            )
+        )
+        .alias("last_wo_null"),
+        f.last_value(col('"Type 2"'))
+        .over(
+            Window(
+                window_frame=WindowFrame("rows", None, 0),
+                order_by=[col('"Speed"')],
+                null_treatment=NullTreatment.RESPECT_NULLS,
+            )
+        )
+        .alias("last_with_null"),
     )
 
 Aggregate Functions
diff --git a/_sources/user-guide/configuration.rst.txt b/_sources/user-guide/configuration.rst.txt
index db200a46..5425a040 100644
--- a/_sources/user-guide/configuration.rst.txt
+++ b/_sources/user-guide/configuration.rst.txt
@@ -46,6 +46,141 @@ a :py:class:`~datafusion.context.SessionConfig` and :py:class:`~datafusion.conte
     ctx = SessionContext(config, runtime)
     print(ctx)
 
+Maximizing CPU Usage
+--------------------
 
-You can read more about available :py:class:`~datafusion.context.SessionConfig` options in the `rust DataFusion Configuration guide <https://arrow.apache.org/datafusion/user-guide/configs.html>`_,
+DataFusion uses partitions to parallelize work. For small queries the
+default configuration (number of CPU cores) is often sufficient, but to
+fully utilize available hardware you can tune how many partitions are
+created and when DataFusion will repartition data automatically.
+
+Configure a ``SessionContext`` with a higher partition count:
+
+.. code-block:: python
+
+    from datafusion import SessionConfig, SessionContext
+
+    # allow up to 16 concurrent partitions
+    config = SessionConfig().with_target_partitions(16)
+    ctx = SessionContext(config)
+
+Automatic repartitioning for joins, aggregations, window functions and
+other operations can be enabled to increase parallelism:
+
+.. code-block:: python
+
+    config = (
+        SessionConfig()
+        .with_target_partitions(16)
+        .with_repartition_joins(True)
+        .with_repartition_aggregations(True)
+        .with_repartition_windows(True)
+    )
+
+Manual repartitioning is available on DataFrames when you need precise
+control:
+
+.. code-block:: python
+
+    from datafusion import col
+
+    df = ctx.read_parquet("data.parquet")
+
+    # Evenly divide into 16 partitions
+    df = df.repartition(16)
+
+    # Or partition by the hash of a column
+    df = df.repartition_by_hash(col("a"), num=16)
+
+    result = df.collect()
+
+
+Benchmark Example
+^^^^^^^^^^^^^^^^^
+
+The repository includes a benchmark script that demonstrates how to maximize CPU usage
+with DataFusion. The :code:`benchmarks/max_cpu_usage.py` script shows a practical example
+of configuring DataFusion for optimal parallelism.
+
+You can run the benchmark script to see the impact of different configuration settings:
+
+.. code-block:: bash
+
+    # Run with default settings (uses all CPU cores)
+    python benchmarks/max_cpu_usage.py
+
+    # Run with specific number of rows and partitions
+    python benchmarks/max_cpu_usage.py --rows 5000000 --partitions 16
+
+    # See all available options
+    python benchmarks/max_cpu_usage.py --help
+
+Here's an example showing the performance difference between single and multiple partitions:
+
+.. code-block:: bash
+
+    # Single partition - slower processing
+    $ python benchmarks/max_cpu_usage.py --rows=10000000 --partitions 1
+    Processed 10000000 rows using 1 partitions in 0.107s
+
+    # Multiple partitions - faster processing
+    $ python benchmarks/max_cpu_usage.py --rows=10000000 --partitions 10
+    Processed 10000000 rows using 10 partitions in 0.038s
+
+This example demonstrates nearly 3x performance improvement (0.107s vs 0.038s) when using 
+10 partitions instead of 1, showcasing how proper partitioning can significantly improve 
+CPU utilization and query performance.
+
+The script demonstrates several key optimization techniques:
+
+1. **Higher target partition count**: Uses :code:`with_target_partitions()` to set the number of concurrent partitions
+2. **Automatic repartitioning**: Enables repartitioning for joins, aggregations, and window functions
+3. **Manual repartitioning**: Uses :code:`repartition()` to ensure all partitions are utilized
+4. **CPU-intensive operations**: Performs aggregations that can benefit from parallelization
+
+The benchmark creates synthetic data and measures the time taken to perform a sum aggregation
+across the specified number of partitions. This helps you understand how partition configuration
+affects performance on your specific hardware.
+
+Important Considerations
+""""""""""""""""""""""""
+
+The provided benchmark script demonstrates partitioning concepts using synthetic in-memory data
+and simple aggregation operations. While useful for understanding basic configuration principles,
+actual performance in production environments may vary significantly based on numerous factors:
+
+**Data Sources and I/O Characteristics:**
+
+- **Table providers**: Performance differs greatly between Parquet files, CSV files, databases, and cloud storage
+- **Storage type**: Local SSD, network-attached storage, and cloud storage have vastly different characteristics  
+- **Network latency**: Remote data sources introduce additional latency considerations
+- **File sizes and distribution**: Large files may benefit differently from partitioning than many small files
+
+**Query and Workload Characteristics:**
+
+- **Operation complexity**: Simple aggregations versus complex joins, window functions, or nested queries
+- **Data distribution**: Skewed data may not partition evenly, affecting parallel efficiency
+- **Memory usage**: Large datasets may require different memory management strategies
+- **Concurrent workloads**: Multiple queries running simultaneously affect resource allocation
+
+**Hardware and Environment Factors:**
+
+- **CPU architecture**: Different processors have varying parallel processing capabilities
+- **Available memory**: Limited RAM may require different optimization strategies
+- **System load**: Other applications competing for resources affect DataFusion performance
+
+**Recommendations for Production Use:**
+
+To optimize DataFusion for your specific use case, it is strongly recommended to:
+
+1. **Create custom benchmarks** using your actual data sources, formats, and query patterns
+2. **Test with representative data volumes** that match your production workloads  
+3. **Measure end-to-end performance** including data loading, processing, and result handling
+4. **Evaluate different configuration combinations** for your specific hardware and workload
+5. **Monitor resource utilization** (CPU, memory, I/O) to identify bottlenecks in your environment
+
+This approach will provide more accurate insights into how DataFusion configuration options
+will impact your particular applications and infrastructure.
+
+For more information about available :py:class:`~datafusion.context.SessionConfig` options, see the `rust DataFusion Configuration guide <https://arrow.apache.org/datafusion/user-guide/configs.html>`_,
 and about :code:`RuntimeEnvBuilder` options in the rust `online API documentation <https://docs.rs/datafusion/latest/datafusion/execution/runtime_env/struct.RuntimeEnvBuilder.html>`_.
diff --git a/_sources/user-guide/data-sources.rst.txt b/_sources/user-guide/data-sources.rst.txt
index 9c95d58e..a9b119b9 100644
--- a/_sources/user-guide/data-sources.rst.txt
+++ b/_sources/user-guide/data-sources.rst.txt
@@ -25,7 +25,7 @@ DataFusion provides a wide variety of ways to get data into a DataFrame to perfo
 Local file
 ----------
 
-DataFusion has the abilty to read from a variety of popular file formats, such as :ref:`Parquet <io_parquet>`,
+DataFusion has the ability to read from a variety of popular file formats, such as :ref:`Parquet <io_parquet>`,
 :ref:`CSV <io_csv>`, :ref:`JSON <io_json>`, and :ref:`AVRO <io_avro>`.
 
 .. ipython:: python
@@ -120,7 +120,7 @@ DataFusion can import DataFrames directly from other libraries, such as
 `Polars <https://pola.rs/>`_ and `Pandas <https://pandas.pydata.org/>`_.
 Since DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule
 interface can be imported to DataFusion using the
-:py:func:`~datafusion.context.SessionContext.from_arrow` function. Older verions of Polars may
+:py:func:`~datafusion.context.SessionContext.from_arrow` function. Older versions of Polars may
 not support the arrow interface. In those cases, you can still import via the
 :py:func:`~datafusion.context.SessionContext.from_polars` function.
 
@@ -172,10 +172,41 @@ which can lead to a significant performance difference.
     df = ctx.table("my_delta_table")
     df.show()
 
-Iceberg
--------
+Apache Iceberg
+--------------
 
-Coming soon!
+DataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface.
+
+This requires either the `pyiceberg <https://pypi.org/project/pyiceberg/>`__ library (>=0.10.0) or the `pyiceberg-core <https://pypi.org/project/pyiceberg-core/>`__ library (>=0.5.0).
+
+* The ``pyiceberg-core`` library exposes Iceberg Rust's implementation of the Custom Table Provider interface as python bindings.
+* The ``pyiceberg`` library utilizes the ``pyiceberg-core`` python bindings under the hood and provides a native way for Python users to interact with the DataFusion.
+
+.. code-block:: python
+
+    from datafusion import SessionContext
+    from pyiceberg.catalog import load_catalog
+    import pyarrow as pa
+
+    # Load catalog and create/load a table
+    catalog = load_catalog("catalog", type="in-memory")
+    catalog.create_namespace_if_not_exists("default")
+
+    # Create some sample data
+    data = pa.table({"x": [1, 2, 3], "y": [4, 5, 6]})
+    iceberg_table = catalog.create_table("default.test", schema=data.schema)
+    iceberg_table.append(data)
+
+    # Register the table with DataFusion
+    ctx = SessionContext()
+    ctx.register_table_provider("test", iceberg_table)
+
+    # Query the table using DataFusion
+    ctx.table("test").show()
+
+
+Note that the Datafusion integration rely on features from the `Iceberg Rust <https://github.com/apache/iceberg-rust/>`_ implementation instead of the `PyIceberg <https://github.com/apache/iceberg-python/>`_ implementation. 
+Features that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion.
 
 Custom Table Provider
 ---------------------
@@ -183,7 +214,7 @@ Custom Table Provider
 You can implement a custom Data Provider in Rust and expose it to DataFusion through the
 the interface as describe in the :ref:`Custom Table Provider <io_custom_table_provider>`
 section. This is an advanced topic, but a
-`user example <https://github.com/apache/datafusion-python/tree/main/examples/ffi-table-provider>`_
+`user example <https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example>`_
 is provided in the DataFusion repository.
 
 Catalog
diff --git a/_sources/user-guide/dataframe/index.rst.txt b/_sources/user-guide/dataframe/index.rst.txt
index f69485af..1387db0b 100644
--- a/_sources/user-guide/dataframe/index.rst.txt
+++ b/_sources/user-guide/dataframe/index.rst.txt
@@ -126,6 +126,56 @@ DataFusion's DataFrame API offers a wide range of operations:
     # Drop columns
     df = df.drop("temporary_column")
 
+Column Names as Function Arguments
+----------------------------------
+
+Some ``DataFrame`` methods accept column names when an argument refers to an
+existing column. These include:
+
+* :py:meth:`~datafusion.DataFrame.select`
+* :py:meth:`~datafusion.DataFrame.sort`
+* :py:meth:`~datafusion.DataFrame.drop`
+* :py:meth:`~datafusion.DataFrame.join` (``on`` argument)
+* :py:meth:`~datafusion.DataFrame.aggregate` (grouping columns)
+
+See the full function documentation for details on any specific function.
+
+Note that :py:meth:`~datafusion.DataFrame.join_on` expects ``col()``/``column()`` expressions rather than plain strings.
+
+For such methods, you can pass column names directly:
+
+.. code-block:: python
+
+    from datafusion import col, functions as f
+
+    df.sort('id')
+    df.aggregate('id', [f.count(col('value'))])
+
+The same operation can also be written with explicit column expressions, using either ``col()`` or ``column()``:
+
+.. code-block:: python
+
+    from datafusion import col, column, functions as f
+
+    df.sort(col('id'))
+    df.aggregate(column('id'), [f.count(col('value'))])
+
+Note that ``column()`` is an alias of ``col()``, so you can use either name; the example above shows both in action.
+
+Whenever an argument represents an expression—such as in
+:py:meth:`~datafusion.DataFrame.filter` or
+:py:meth:`~datafusion.DataFrame.with_column`—use ``col()`` to reference
+columns. The comparison and arithmetic operators on ``Expr`` will automatically
+convert any non-``Expr`` value into a literal expression, so writing
+
+.. code-block:: python
+
+    from datafusion import col
+    df.filter(col("age") > 21)
+
+is equivalent to using ``lit(21)`` explicitly. Use ``lit()`` (also available
+as ``literal()``) when you need to construct a literal expression directly.
+
 Terminal Operations
 -------------------
 
diff --git a/autoapi/datafusion/context/index.html b/autoapi/datafusion/context/index.html
index 5da05964..6ab8fbc0 100644
--- a/autoapi/datafusion/context/index.html
+++ b/autoapi/datafusion/context/index.html
@@ -771,6 +771,15 @@
        </code>
       </a>
      </li>
+     <li class="toc-h4 nav-item toc-entry">
+      <a class="reference internal nav-link" href="#datafusion.context.SessionContext._convert_file_sort_order">
+       <code class="docutils literal notranslate">
+        <span class="pre">
+         SessionContext._convert_file_sort_order()
+        </span>
+       </code>
+      </a>
+     </li>
      <li class="toc-h4 nav-item toc-entry">
       <a class="reference internal nav-link" href="#datafusion.context.SessionContext._convert_table_partition_cols">
        <code class="docutils literal notranslate">
@@ -1734,6 +1743,15 @@ a query against a CSV data source using the <code class="xref py py-class docuti
 <dd><p>Print a string representation of the Session Context.</p>
 </dd></dl>
 
+<dl class="py method">
+<dt class="sig sig-object py" id="datafusion.context.SessionContext._convert_file_sort_order">
+<em class="property"><span class="pre">static</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">_convert_file_sort_order</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file_sort_order</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">collections.abc.Sequence</span><span class="p"><span class="pre">[</span></span><span class="pre">c [...]
+<dd><p>Convert nested <code class="docutils literal notranslate"><span class="pre">SortKey</span></code> sequences into raw sort expressions.</p>
+<p>Each <code class="docutils literal notranslate"><span class="pre">SortKey</span></code> can be a column name string, an <code class="docutils literal notranslate"><span class="pre">Expr</span></code>, or a
+<code class="docutils literal notranslate"><span class="pre">SortExpr</span></code> and will be converted using
+<code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.expr.sort_list_to_raw_sort_list()</span></code>.</p>
+</dd></dl>
+
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.context.SessionContext._convert_table_partition_cols">
 <em class="property"><span class="pre">static</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">_convert_table_partition_cols</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">table_partition_cols</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><span class="pre">tuple</span [...]
@@ -1991,7 +2009,7 @@ selected for data input.</p></li>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.context.SessionContext.read_parquet">
-<span class="sig-name descname"><span class="pre">read_parquet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">pathlib.Path</span></span></em>, <em class="sig-param"><span class="n"><span c [...]
+<span class="sig-name descname"><span class="pre">read_parquet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span><span class="w"> </span><span class="p"><span class="pre">|</span></span><span class="w"> </span><span class="pre">pathlib.Path</span></span></em>, <em class="sig-param"><span class="n"><span c [...]
 <dd><p>Read a Parquet source into a <code class="xref py py-class docutils literal notranslate"><span class="pre">Dataframe</span></code>.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
@@ -2008,7 +2026,9 @@ conflicts due to metadata.</p></li>
 <li><p><strong>schema</strong> – An optional schema representing the parquet files. If None,
 the parquet reader will try to infer it based on data in the
 file.</p></li>
-<li><p><strong>file_sort_order</strong> – Sort order for the file.</p></li>
+<li><p><strong>file_sort_order</strong> – Sort order for the file. Each sort key can be
+specified as a column name (<code class="docutils literal notranslate"><span class="pre">str</span></code>), an expression
+(<code class="docutils literal notranslate"><span class="pre">Expr</span></code>), or a <code class="docutils literal notranslate"><span class="pre">SortExpr</span></code>.</p></li>
 </ul>
 </dd>
 <dt class="field-even">Returns<span class="colon">:</span></dt>
@@ -2116,7 +2136,7 @@ selected for data input.</p></li>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.context.SessionContext.register_listing_table">
-<span class="sig-name descname"><span class="pre">register_listing_table</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span  [...]
+<span class="sig-name descname"><span class="pre">register_listing_table</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span  [...]
 <dd><p>Register multiple files as a single table.</p>
 <p>Registers a <a class="reference internal" href="../catalog/index.html#datafusion.catalog.Table" title="datafusion.catalog.Table"><code class="xref py py-class docutils literal notranslate"><span class="pre">Table</span></code></a> that can assemble multiple
 files from locations in an <code class="xref py py-class docutils literal notranslate"><span class="pre">ObjectStore</span></code>
@@ -2129,7 +2149,9 @@ instance.</p>
 <li><p><strong>table_partition_cols</strong> – Partition columns.</p></li>
 <li><p><strong>file_extension</strong> – File extension of the provided table.</p></li>
 <li><p><strong>schema</strong> – The data source schema.</p></li>
-<li><p><strong>file_sort_order</strong> – Sort order for the file.</p></li>
+<li><p><strong>file_sort_order</strong> – Sort order for the file. Each sort key can be
+specified as a column name (<code class="docutils literal notranslate"><span class="pre">str</span></code>), an expression
+(<code class="docutils literal notranslate"><span class="pre">Expr</span></code>), or a <code class="docutils literal notranslate"><span class="pre">SortExpr</span></code>.</p></li>
 </ul>
 </dd>
 </dl>
@@ -2152,7 +2174,7 @@ instance.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.context.SessionContext.register_parquet">
-<span class="sig-name descname"><span class="pre">register_parquet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class= [...]
+<span class="sig-name descname"><span class="pre">register_parquet</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class= [...]
 <dd><p>Register a Parquet file as a table.</p>
 <p>The registered table can be referenced from SQL statement executed
 against this context.</p>
@@ -2170,7 +2192,9 @@ selected for data input.</p></li>
 that may be in the file schema. This can help avoid schema
 conflicts due to metadata.</p></li>
 <li><p><strong>schema</strong> – The data source schema.</p></li>
-<li><p><strong>file_sort_order</strong> – Sort order for the file.</p></li>
+<li><p><strong>file_sort_order</strong> – Sort order for the file. Each sort key can be
+specified as a column name (<code class="docutils literal notranslate"><span class="pre">str</span></code>), an expression
+(<code class="docutils literal notranslate"><span class="pre">Expr</span></code>), or a <code class="docutils literal notranslate"><span class="pre">SortExpr</span></code>.</p></li>
 </ul>
 </dd>
 </dl>
diff --git a/autoapi/datafusion/dataframe/index.html b/autoapi/datafusion/dataframe/index.html
index 965fe819..3c61a0aa 100644
--- a/autoapi/datafusion/dataframe/index.html
+++ b/autoapi/datafusion/dataframe/index.html
@@ -1015,42 +1015,6 @@
      </code>
     </a>
     <ul class="nav section-nav flex-column">
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.encoding">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.encoding
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.dictionary_enabled">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.dictionary_enabled
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.compression">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.compression
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.statistics_enabled">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.statistics_enabled
-        </span>
-       </code>
-      </a>
-     </li>
      <li class="toc-h4 nav-item toc-entry">
       <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.bloom_filter_enabled">
        <code class="docutils literal notranslate">
@@ -1079,34 +1043,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id0">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.bloom_filter_enabled
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id1">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.bloom_filter_fpp
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id2">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.bloom_filter_ndv
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id3">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.compression">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetColumnOptions.compression
@@ -1115,7 +1052,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id4">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.dictionary_enabled">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetColumnOptions.dictionary_enabled
@@ -1124,7 +1061,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id5">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.encoding">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetColumnOptions.encoding
@@ -1133,7 +1070,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id6">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetColumnOptions.statistics_enabled">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetColumnOptions.statistics_enabled
@@ -1152,159 +1089,6 @@
      </code>
     </a>
     <ul class="nav section-nav flex-column">
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.data_pagesize_limit">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.data_pagesize_limit
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.write_batch_size">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.write_batch_size
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.writer_version">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.writer_version
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.skip_arrow_metadata">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.skip_arrow_metadata
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.compression">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.compression
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.dictionary_enabled">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.dictionary_enabled
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.dictionary_page_size_limit">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.dictionary_page_size_limit
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.statistics_enabled">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.statistics_enabled
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.max_row_group_size">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.max_row_group_size
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.created_by">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.created_by
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.column_index_truncate_length">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.column_index_truncate_length
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.statistics_truncate_length">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.statistics_truncate_length
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.data_page_row_count_limit">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.data_page_row_count_limit
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.encoding">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.encoding
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_on_write">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.bloom_filter_on_write
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_fpp">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.bloom_filter_fpp
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_ndv">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.bloom_filter_ndv
-        </span>
-       </code>
-      </a>
-     </li>
      <li class="toc-h4 nav-item toc-entry">
       <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.allow_single_file_parallelism">
        <code class="docutils literal notranslate">
@@ -1315,43 +1099,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.maximum_parallel_row_group_writers">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.maximum_parallel_row_group_writers
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.maximum_buffered_record_batches_per_stream
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.column_specific_options">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.column_specific_options
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id7">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.allow_single_file_parallelism
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id8">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_fpp">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.bloom_filter_fpp
@@ -1360,7 +1108,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id9">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_ndv">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.bloom_filter_ndv
@@ -1369,7 +1117,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id10">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_on_write">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.bloom_filter_on_write
@@ -1378,7 +1126,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id11">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.column_index_truncate_length">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.column_index_truncate_length
@@ -1387,7 +1135,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id12">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.column_specific_options">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.column_specific_options
@@ -1396,7 +1144,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id13">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.created_by">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.created_by
@@ -1405,7 +1153,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id14">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.data_page_row_count_limit">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.data_page_row_count_limit
@@ -1414,7 +1162,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id15">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.data_pagesize_limit">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.data_pagesize_limit
@@ -1423,7 +1171,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id16">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.dictionary_enabled">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.dictionary_enabled
@@ -1432,7 +1180,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id17">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.dictionary_page_size_limit">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.dictionary_page_size_limit
@@ -1441,7 +1189,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id18">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.encoding">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.encoding
@@ -1450,7 +1198,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id19">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.max_row_group_size">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.max_row_group_size
@@ -1459,7 +1207,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id20">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.maximum_buffered_record_batches_per_stream
@@ -1468,7 +1216,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id21">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.maximum_parallel_row_group_writers">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.maximum_parallel_row_group_writers
@@ -1477,7 +1225,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id22">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.skip_arrow_metadata">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.skip_arrow_metadata
@@ -1486,7 +1234,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id23">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.statistics_enabled">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.statistics_enabled
@@ -1495,7 +1243,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id24">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.statistics_truncate_length">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.statistics_truncate_length
@@ -1504,7 +1252,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id25">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.write_batch_size">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.write_batch_size
@@ -1513,7 +1261,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id26">
+      <a class="reference internal nav-link" href="#datafusion.dataframe.ParquetWriterOptions.writer_version">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.writer_version
@@ -1699,13 +1447,13 @@ those in the DataFrame.</p>
 
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.dataframe.DataFrame.aggregate">
-<span class="sig-name descname"><span class="pre">aggregate</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">group_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.expr.Ex [...]
+<span class="sig-name descname"><span class="pre">aggregate</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">group_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">collections.abc.Sequence</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre [...]
 <dd><p>Aggregates the rows of the current DataFrame.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
-<li><p><strong>group_by</strong> – List of expressions to group by.</p></li>
-<li><p><strong>aggs</strong> – List of expressions to aggregate.</p></li>
+<li><p><strong>group_by</strong> – Sequence of expressions or column names to group by.</p></li>
+<li><p><strong>aggs</strong> – Sequence of expressions to aggregate.</p></li>
 </ul>
 </dd>
 <dt class="field-even">Returns<span class="colon">:</span></dt>
@@ -1743,7 +1491,7 @@ those in the DataFrame.</p>
 <dt class="sig sig-object py" id="datafusion.dataframe.DataFrame.collect">
 <span class="sig-name descname"><span class="pre">collect</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><span class="pre">pyarrow.RecordBatch</span><span class="p"><span class="pre">]</span></span></span></span><a class="headerlink" href="#datafusion.dataframe.DataFrame.collec [...]
 <dd><p>Execute this <a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">DataFrame</span></code></a> and collect results into memory.</p>
-<p>Prior to calling <code class="docutils literal notranslate"><span class="pre">collect</span></code>, modifying a DataFrme simply updates a plan
+<p>Prior to calling <code class="docutils literal notranslate"><span class="pre">collect</span></code>, modifying a DataFrame simply updates a plan
 (no actual computation is performed). Calling <code class="docutils literal notranslate"><span class="pre">collect</span></code> triggers the
 computation.</p>
 <dl class="field-list simple">
@@ -1887,7 +1635,7 @@ for non-numeric datatypes.</p>
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>verbose</strong> – If <code class="docutils literal notranslate"><span class="pre">True</span></code>, more details will be included.</p></li>
-<li><p><strong>analyze</strong> – If <code class="docutils literal notranslate"><span class="pre">Tru`e</span></code>, the plan will run and metrics reported.</p></li>
+<li><p><strong>analyze</strong> – If <code class="docutils literal notranslate"><span class="pre">True</span></code>, the plan will run and metrics reported.</p></li>
 </ul>
 </dd>
 </dl>
@@ -1927,9 +1675,17 @@ for non-numeric datatypes.</p>
 <span class="sig-name descname"><span class="pre">filter</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">predicates</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.expr.Expr</span></a></span></em><span [...]
 <dd><p>Return a DataFrame for which <code class="docutils literal notranslate"><span class="pre">predicate</span></code> evaluates to <code class="docutils literal notranslate"><span class="pre">True</span></code>.</p>
 <p>Rows for which <code class="docutils literal notranslate"><span class="pre">predicate</span></code> evaluates to <code class="docutils literal notranslate"><span class="pre">False</span></code> or <code class="docutils literal notranslate"><span class="pre">None</span></code> are filtered
-out.  If more than one predicate is provided, these predicates will be
-combined as a logical AND. If more complex logic is required, see the
-logical operations in <a class="reference internal" href="../functions/index.html#module-datafusion.functions" title="datafusion.functions"><code class="xref py py-mod docutils literal notranslate"><span class="pre">functions</span></code></a>.</p>
+out. If more than one predicate is provided, these predicates will be
+combined as a logical AND. Each <code class="docutils literal notranslate"><span class="pre">predicate</span></code> must be an
+<a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><code class="xref py py-class docutils literal notranslate"><span class="pre">Expr</span></code></a> created using helper functions such as
+<a class="reference internal" href="../index.html#datafusion.col" title="datafusion.col"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.col()</span></code></a> or <a class="reference internal" href="../index.html#datafusion.lit" title="datafusion.lit"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.lit()</span></code></a>.
+If more complex logic is required, see the logical operations in
+<a class="reference internal" href="../functions/index.html#module-datafusion.functions" title="datafusion.functions"><code class="xref py py-mod docutils literal notranslate"><span class="pre">functions</span></code></a>.</p>
+<p>Example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span><span class="p">,</span> <span class="n">lit</span>
+<span class="n">df</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">)</span> <span class="o">&gt;</span> <span class="n">lit</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
+</pre></div>
+</div>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><p><strong>predicates</strong> – Predicate expression(s) to filter the DataFrame.</p>
@@ -1977,9 +1733,9 @@ logical operations in <a class="reference internal" href="../functions/index.htm
 
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.dataframe.DataFrame.join">
-<span class="sig-name descname"><span class="pre">join</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">right</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><span class="pre">DataFrame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">on</span></s [...]
+<span class="sig-name descname"><span class="pre">join</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">right</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><span class="pre">DataFrame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">on</span></s [...]
 <dt class="sig sig-object py">
-<span class="sig-name descname"><span class="pre">join</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">right</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><span class="pre">DataFrame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">on</span></s [...]
+<span class="sig-name descname"><span class="pre">join</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">right</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><span class="pre">DataFrame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">on</span></s [...]
 <dt class="sig sig-object py">
 <span class="sig-name descname"><span class="pre">join</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">right</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><span class="pre">DataFrame</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">on</span></s [...]
 <dd><p>Join this <a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">DataFrame</span></code></a> with another <a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">DataFrame</span></code></a>.</p>
@@ -2006,8 +1762,14 @@ logical operations in <a class="reference internal" href="../functions/index.htm
 <dt class="sig sig-object py" id="datafusion.dataframe.DataFrame.join_on">
 <span class="sig-name descname"><span class="pre">join_on</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">right</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><span class="pre">DataFrame</span></a></span></em>, <em class="sig-param"><span class="o"><span class="pre">*</span>< [...]
 <dd><p>Join two <a class="reference internal" href="#datafusion.dataframe.DataFrame" title="datafusion.dataframe.DataFrame"><code class="xref py py-class docutils literal notranslate"><span class="pre">DataFrame</span></code></a> using the specified expressions.</p>
-<p>On expressions are used to support in-equality predicates. Equality
-predicates are correctly optimized</p>
+<p>Join predicates must be <a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><code class="xref py py-class docutils literal notranslate"><span class="pre">Expr</span></code></a> objects, typically
+built with <a class="reference internal" href="../index.html#datafusion.col" title="datafusion.col"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.col()</span></code></a>. On expressions are used to support
+in-equality predicates. Equality predicates are correctly optimized.</p>
+<p>Example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span>
+<span class="n">df</span><span class="o">.</span><span class="n">join_on</span><span class="p">(</span><span class="n">other_df</span><span class="p">,</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;id&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;other_id&quot;</span><span class="p">))</span>
+</pre></div>
+</div>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
@@ -2153,13 +1915,13 @@ column <code class="docutils literal notranslate"><span class="pre">a</span></co
 
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.dataframe.DataFrame.sort">
-<span class="sig-name descname"><span class="pre">sort</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">exprs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.expr.Expr</span></a><span class="w"> </span> [...]
-<dd><p>Sort the DataFrame by the specified sorting expressions.</p>
+<span class="sig-name descname"><span class="pre">sort</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">exprs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">datafusion.expr.SortKey</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="si [...]
+<dd><p>Sort the DataFrame by the specified sorting expressions or column names.</p>
 <p>Note that any expression can be turned into a sort expression by
-calling its` <code class="docutils literal notranslate"><span class="pre">sort</span></code> method.</p>
+calling its <code class="docutils literal notranslate"><span class="pre">sort</span></code> method.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
-<dd class="field-odd"><p><strong>exprs</strong> – Sort expressions, applied in order.</p>
+<dd class="field-odd"><p><strong>exprs</strong> – Sort expressions or column names, applied in order.</p>
 </dd>
 <dt class="field-even">Returns<span class="colon">:</span></dt>
 <dd class="field-even"><p>DataFrame after sorting.</p>
@@ -2324,6 +2086,13 @@ returned.</p></li>
 <dt class="sig sig-object py" id="datafusion.dataframe.DataFrame.with_column">
 <span class="sig-name descname"><span class="pre">with_column</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">expr</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="referen [...]
 <dd><p>Add an additional column to the DataFrame.</p>
+<p>The <code class="docutils literal notranslate"><span class="pre">expr</span></code> must be an <a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><code class="xref py py-class docutils literal notranslate"><span class="pre">Expr</span></code></a> constructed with
+<a class="reference internal" href="../index.html#datafusion.col" title="datafusion.col"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.col()</span></code></a> or <a class="reference internal" href="../index.html#datafusion.lit" title="datafusion.lit"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.lit()</span></code></a>.</p>
+<p>Example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span><span class="p">,</span> <span class="n">lit</span>
+<span class="n">df</span><span class="o">.</span><span class="n">with_column</span><span class="p">(</span><span class="s2">&quot;b&quot;</span><span class="p">,</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">)</span> <span class="o">+</span> <span class="n">lit</span><span class="p">(</span><span class="mi">1</span><span class="p">))</span>
+</pre></div>
+</div>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
@@ -2361,14 +2130,18 @@ into one the following symbols (” or ‘ or `).</p>
 <dt class="sig sig-object py" id="datafusion.dataframe.DataFrame.with_columns">
 <span class="sig-name descname"><span class="pre">with_columns</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="o"><span class="pre">*</span></span><span class="n"><span class="pre">exprs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.expr.Expr</span></a><span class="w"> [...]
 <dd><p>Add columns to the DataFrame.</p>
-<p>By passing expressions, iteratables of expressions, or named expressions. To
-pass named expressions use the form name=Expr.</p>
-<p>Example usage: The following will add 4 columns labeled a, b, c, and d:</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">with_columns</span><span class="p">(</span>
-    <span class="n">lit</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s1">&#39;a&#39;</span><span class="p">),</span>
-    <span class="p">[</span><span class="n">lit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s1">&#39;b&#39;</span><span class="p">),</span> <span class="n">lit</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s1">&#39;c&#39;</span><span class="p">)] [...]
+<p>By passing expressions, iterables of expressions, or named expressions.
+All expressions must be <a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><code class="xref py py-class docutils literal notranslate"><span class="pre">Expr</span></code></a> objects created via
+<a class="reference internal" href="../index.html#datafusion.col" title="datafusion.col"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.col()</span></code></a> or <a class="reference internal" href="../index.html#datafusion.lit" title="datafusion.lit"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.lit()</span></code></a>.
+To pass named expressions use the form <code class="docutils literal notranslate"><span class="pre">name=Expr</span></code>.</p>
+<p>Example usage: The following will add 4 columns labeled <code class="docutils literal notranslate"><span class="pre">a</span></code>, <code class="docutils literal notranslate"><span class="pre">b</span></code>, <code class="docutils literal notranslate"><span class="pre">c</span></code>,
+and <code class="docutils literal notranslate"><span class="pre">d</span></code>:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span><span class="p">,</span> <span class="n">lit</span>
+<span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">with_columns</span><span class="p">(</span>
+    <span class="n">col</span><span class="p">(</span><span class="s2">&quot;x&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span>
+    <span class="p">[</span><span class="n">lit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;b&quot;</span><span class="p">),</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;y&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;c&quot;</span><s [...]
     <span class="n">d</span><span class="o">=</span><span class="n">lit</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
-    <span class="p">)</span>
+<span class="p">)</span>
 </pre></div>
 </div>
 <dl class="field-list simple">
@@ -2466,96 +2239,68 @@ provide better compression but slower speed.</p></li>
 <em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.dataframe.</span></span><span class="sig-name descname"><span class="pre">ParquetColumnOptions</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span><span cl [...]
 <dd><p>Parquet options for individual columns.</p>
 <p>Contains the available options that can be applied for an individual Parquet column,
-replacing the global options in <cite>ParquetWriterOptions</cite>.</p>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.encoding">
-<span class="sig-name descname"><span class="pre">encoding</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.encoding" title="Link to this definition">¶</a></dt>
-<dd><p>Sets encoding for the column path. Valid values are: <cite>plain</cite>,
-<cite>plain_dictionary</cite>, <cite>rle</cite>, <cite>bit_packed</cite>, <cite>delta_binary_packed</cite>,
-<cite>delta_length_byte_array</cite>, <cite>delta_byte_array</cite>, <cite>rle_dictionary</cite>, and
-<cite>byte_stream_split</cite>. These values are not case-sensitive. If <cite>None</cite>, uses
-the default parquet options</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.dictionary_enabled">
-<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.dictionary_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if dictionary encoding is enabled for the column path.
-If <cite>None</cite>, uses the default parquet options</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.compression">
-<span class="sig-name descname"><span class="pre">compression</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.compression" title="Link to this definition">¶</a></dt>
-<dd><p>Sets default parquet compression codec for the column path. Valid
-values are <cite>uncompressed</cite>, <cite>snappy</cite>, <cite>gzip(level)</cite>, <cite>lzo</cite>, <cite>brotli(level)</cite>,
-<cite>lz4</cite>, <cite>zstd(level)</cite>, and <cite>lz4_raw</cite>. These values are not case-sensitive. If
-<cite>None</cite>, uses the default parquet options.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.statistics_enabled">
-<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.statistics_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if statistics are enabled for the column Valid values
-are: <cite>none</cite>, <cite>chunk</cite>, and <cite>page</cite> These values are not case sensitive. If
-<cite>None</cite>, uses the default parquet options.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.bloom_filter_enabled">
-<span class="sig-name descname"><span class="pre">bloom_filter_enabled</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.bloom_filter_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if bloom filter is enabled for the column path. If
-<cite>None</cite>, uses the default parquet options.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.bloom_filter_fpp">
-<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.bloom_filter_fpp" title="Link to this definition">¶</a></dt>
-<dd><p>Sets bloom filter false positive probability for the column
-path. If <cite>None</cite>, uses the default parquet options.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.bloom_filter_ndv">
-<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.bloom_filter_ndv" title="Link to this definition">¶</a></dt>
-<dd><p>Sets bloom filter number of distinct values. If <cite>None</cite>, uses
-the default parquet options.</p>
-</dd></dl>
-
+replacing the global options in <code class="docutils literal notranslate"><span class="pre">ParquetWriterOptions</span></code>.</p>
 <p>Initialize the ParquetColumnOptions.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>encoding</strong> – Sets encoding for the column path. Valid values are: <code class="docutils literal notranslate"><span class="pre">plain</span></code>,
+<code class="docutils literal notranslate"><span class="pre">plain_dictionary</span></code>, <code class="docutils literal notranslate"><span class="pre">rle</span></code>, <code class="docutils literal notranslate"><span class="pre">bit_packed</span></code>, <code class="docutils literal notranslate"><span class="pre">delta_binary_packed</span></code>,
+<code class="docutils literal notranslate"><span class="pre">delta_length_byte_array</span></code>, <code class="docutils literal notranslate"><span class="pre">delta_byte_array</span></code>, <code class="docutils literal notranslate"><span class="pre">rle_dictionary</span></code>,
+and <code class="docutils literal notranslate"><span class="pre">byte_stream_split</span></code>. These values are not case-sensitive. If
+<code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet options</p></li>
+<li><p><strong>dictionary_enabled</strong> – Sets if dictionary encoding is enabled for the column
+path. If <cite>None</cite>, uses the default parquet options</p></li>
+<li><p><strong>compression</strong> – Sets default parquet compression codec for the column path.
+Valid values are <code class="docutils literal notranslate"><span class="pre">uncompressed</span></code>, <code class="docutils literal notranslate"><span class="pre">snappy</span></code>, <code class="docutils literal notranslate"><span class="pre">gzip(level)</span></code>, <code class="docutils literal notranslate"><span class="pre">lzo</span></code>,
+<code class="docutils literal notranslate"><span class="pre">brotli(level)</span></code>, <code class="docutils literal notranslate"><span class="pre">lz4</span></code>, <code class="docutils literal notranslate"><span class="pre">zstd(level)</span></code>, and <code class="docutils literal notranslate"><span class="pre">lz4_raw</span></code>. These
+values are not case-sensitive. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet
+options.</p></li>
+<li><p><strong>statistics_enabled</strong> – Sets if statistics are enabled for the column Valid
+values are: <code class="docutils literal notranslate"><span class="pre">none</span></code>, <code class="docutils literal notranslate"><span class="pre">chunk</span></code>, and <code class="docutils literal notranslate"><span class="pre">page</span></code> These values are not case
+sensitive. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet options.</p></li>
+<li><p><strong>bloom_filter_enabled</strong> – Sets if bloom filter is enabled for the column path.
+If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet options.</p></li>
+<li><p><strong>bloom_filter_fpp</strong> – Sets bloom filter false positive probability for the
+column path. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet options.</p></li>
+<li><p><strong>bloom_filter_ndv</strong> – Sets bloom filter number of distinct values. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet options.</p></li>
+</ul>
+</dd>
+</dl>
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id0">
-<span class="sig-name descname"><span class="pre">bloom_filter_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id0" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.bloom_filter_enabled">
+<span class="sig-name descname"><span class="pre">bloom_filter_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.bloom_filter_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id1">
-<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id1" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.bloom_filter_fpp">
+<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.bloom_filter_fpp" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id2">
-<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id2" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.bloom_filter_ndv">
+<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.bloom_filter_ndv" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id3">
-<span class="sig-name descname"><span class="pre">compression</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id3" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.compression">
+<span class="sig-name descname"><span class="pre">compression</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.compression" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id4">
-<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id4" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.dictionary_enabled">
+<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.dictionary_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id5">
-<span class="sig-name descname"><span class="pre">encoding</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id5" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.encoding">
+<span class="sig-name descname"><span class="pre">encoding</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.encoding" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id6">
-<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id6" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetColumnOptions.statistics_enabled">
+<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetColumnOptions.statistics_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 </dd></dl>
@@ -2565,270 +2310,177 @@ the default parquet options.</p>
 <em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.dataframe.</span></span><span class="sig-name descname"><span class="pre">ParquetWriterOptions</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">data_pagesize_limit</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</sp [...]
 <dd><p>Advanced parquet writer options.</p>
 <p>Allows settings the writer options that apply to the entire file. Some options can
-also be set on a column by column basis, with the field <cite>column_specific_options</cite>
-(see <cite>ParquetColumnOptions</cite>).</p>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.data_pagesize_limit">
-<span class="sig-name descname"><span class="pre">data_pagesize_limit</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.data_pagesize_limit" title="Link to this definition">¶</a></dt>
-<dd><p>Sets best effort maximum size of data page in bytes.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.write_batch_size">
-<span class="sig-name descname"><span class="pre">write_batch_size</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.write_batch_size" title="Link to this definition">¶</a></dt>
-<dd><p>Sets write_batch_size in bytes.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.writer_version">
-<span class="sig-name descname"><span class="pre">writer_version</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.writer_version" title="Link to this definition">¶</a></dt>
-<dd><p>Sets parquet writer version. Valid values are <cite>1.0</cite> and
-<cite>2.0</cite>.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.skip_arrow_metadata">
-<span class="sig-name descname"><span class="pre">skip_arrow_metadata</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.skip_arrow_metadata" title="Link to this definition">¶</a></dt>
-<dd><p>Skip encoding the embedded arrow metadata in the
-KV_meta.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.compression">
-<span class="sig-name descname"><span class="pre">compression</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.compression" title="Link to this definition">¶</a></dt>
-<dd><p>Compression type to use. Default is “zstd(3)”.
-Available compression types are
-- “uncompressed”: No compression.
-- “snappy”: Snappy compression.
-- “gzip(n)”: Gzip compression with level n.
-- “brotli(n)”: Brotli compression with level n.
-- “lz4”: LZ4 compression.
-- “lz4_raw”: LZ4_RAW compression.
-- “zstd(n)”: Zstandard compression with level n.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.dictionary_enabled">
-<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.dictionary_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if dictionary encoding is enabled. If None, uses
-the default parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.dictionary_page_size_limit">
-<span class="sig-name descname"><span class="pre">dictionary_page_size_limit</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.dictionary_page_size_limit" title="Link to this definition">¶</a></dt>
-<dd><p>Sets best effort maximum dictionary page size,
-in bytes.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.statistics_enabled">
-<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.statistics_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if statistics are enabled for any column Valid
-values are <cite>none</cite>, <cite>chunk</cite>, and <cite>page</cite>. If None, uses the default
-parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.max_row_group_size">
-<span class="sig-name descname"><span class="pre">max_row_group_size</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.max_row_group_size" title="Link to this definition">¶</a></dt>
-<dd><p>Target maximum number of rows in each row group
-(defaults to 1M rows). Writing larger row groups requires more memory to
-write, but can get better compression and be faster to read.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.created_by">
-<span class="sig-name descname"><span class="pre">created_by</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.created_by" title="Link to this definition">¶</a></dt>
-<dd><p>Sets “created by” property.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.column_index_truncate_length">
-<span class="sig-name descname"><span class="pre">column_index_truncate_length</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.column_index_truncate_length" title="Link to this definition">¶</a></dt>
-<dd><p>Sets column index truncate length.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.statistics_truncate_length">
-<span class="sig-name descname"><span class="pre">statistics_truncate_length</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.statistics_truncate_length" title="Link to this definition">¶</a></dt>
-<dd><p>Sets statistics truncate length. If None, uses
-the default parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.data_page_row_count_limit">
-<span class="sig-name descname"><span class="pre">data_page_row_count_limit</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.data_page_row_count_limit" title="Link to this definition">¶</a></dt>
-<dd><p>Sets best effort maximum number of rows in a data
-page.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.encoding">
-<span class="sig-name descname"><span class="pre">encoding</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.encoding" title="Link to this definition">¶</a></dt>
-<dd><p>Sets default encoding for any column. Valid values are <cite>plain</cite>,
-<cite>plain_dictionary</cite>, <cite>rle</cite>, <cite>bit_packed</cite>, <cite>delta_binary_packed</cite>,
-<cite>delta_length_byte_array</cite>, <cite>delta_byte_array</cite>, <cite>rle_dictionary</cite>, and
-<cite>byte_stream_split</cite>. If None, uses the default parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.bloom_filter_on_write">
-<span class="sig-name descname"><span class="pre">bloom_filter_on_write</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_on_write" title="Link to this definition">¶</a></dt>
-<dd><p>Write bloom filters for all columns when creating
-parquet files.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.bloom_filter_fpp">
-<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_fpp" title="Link to this definition">¶</a></dt>
-<dd><p>Sets bloom filter false positive probability. If None,
-uses the default parquet writer setting</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.bloom_filter_ndv">
-<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_ndv" title="Link to this definition">¶</a></dt>
-<dd><p>Sets bloom filter number of distinct values. If None, uses
-the default parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.allow_single_file_parallelism">
-<span class="sig-name descname"><span class="pre">allow_single_file_parallelism</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.allow_single_file_parallelism" title="Link to this definition">¶</a></dt>
-<dd><p>Controls whether DataFusion will attempt to
+also be set on a column by column basis, with the field <code class="docutils literal notranslate"><span class="pre">column_specific_options</span></code>
+(see <code class="docutils literal notranslate"><span class="pre">ParquetColumnOptions</span></code>).</p>
+<p>Initialize the ParquetWriterOptions.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>data_pagesize_limit</strong> – Sets best effort maximum size of data page in bytes.</p></li>
+<li><p><strong>write_batch_size</strong> – Sets write_batch_size in bytes.</p></li>
+<li><p><strong>writer_version</strong> – Sets parquet writer version. Valid values are <code class="docutils literal notranslate"><span class="pre">1.0</span></code> and
+<code class="docutils literal notranslate"><span class="pre">2.0</span></code>.</p></li>
+<li><p><strong>skip_arrow_metadata</strong> – Skip encoding the embedded arrow metadata in the
+KV_meta.</p></li>
+<li><p><strong>compression</strong> – <p>Compression type to use. Default is <code class="docutils literal notranslate"><span class="pre">zstd(3)</span></code>.
+Available compression types are</p>
+<ul>
+<li><p><code class="docutils literal notranslate"><span class="pre">uncompressed</span></code>: No compression.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">snappy</span></code>: Snappy compression.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">gzip(n)</span></code>: Gzip compression with level n.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">brotli(n)</span></code>: Brotli compression with level n.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">lz4</span></code>: LZ4 compression.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">lz4_raw</span></code>: LZ4_RAW compression.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">zstd(n)</span></code>: Zstandard compression with level n.</p></li>
+</ul>
+</p></li>
+<li><p><strong>compression_level</strong> – Compression level to set.</p></li>
+<li><p><strong>dictionary_enabled</strong> – Sets if dictionary encoding is enabled. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet writer setting.</p></li>
+<li><p><strong>dictionary_page_size_limit</strong> – Sets best effort maximum dictionary page size,
+in bytes.</p></li>
+<li><p><strong>statistics_enabled</strong> – Sets if statistics are enabled for any column Valid
+values are <code class="docutils literal notranslate"><span class="pre">none</span></code>, <code class="docutils literal notranslate"><span class="pre">chunk</span></code>, and <code class="docutils literal notranslate"><span class="pre">page</span></code>. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the
+default parquet writer setting.</p></li>
+<li><p><strong>max_row_group_size</strong> – Target maximum number of rows in each row group
+(defaults to 1M rows). Writing larger row groups requires more memory
+to write, but can get better compression and be faster to read.</p></li>
+<li><p><strong>created_by</strong> – Sets “created by” property.</p></li>
+<li><p><strong>column_index_truncate_length</strong> – Sets column index truncate length.</p></li>
+<li><p><strong>statistics_truncate_length</strong> – Sets statistics truncate length. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet writer setting.</p></li>
+<li><p><strong>data_page_row_count_limit</strong> – Sets best effort maximum number of rows in a data
+page.</p></li>
+<li><p><strong>encoding</strong> – Sets default encoding for any column. Valid values are <code class="docutils literal notranslate"><span class="pre">plain</span></code>,
+<code class="docutils literal notranslate"><span class="pre">plain_dictionary</span></code>, <code class="docutils literal notranslate"><span class="pre">rle</span></code>, <code class="docutils literal notranslate"><span class="pre">bit_packed</span></code>, <code class="docutils literal notranslate"><span class="pre">delta_binary_packed</span></code>,
+<code class="docutils literal notranslate"><span class="pre">delta_length_byte_array</span></code>, <code class="docutils literal notranslate"><span class="pre">delta_byte_array</span></code>, <code class="docutils literal notranslate"><span class="pre">rle_dictionary</span></code>,
+and <code class="docutils literal notranslate"><span class="pre">byte_stream_split</span></code>. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet writer
+setting.</p></li>
+<li><p><strong>bloom_filter_on_write</strong> – Write bloom filters for all columns when creating
+parquet files.</p></li>
+<li><p><strong>bloom_filter_fpp</strong> – Sets bloom filter false positive probability. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet writer setting</p></li>
+<li><p><strong>bloom_filter_ndv</strong> – Sets bloom filter number of distinct values. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet writer setting.</p></li>
+<li><p><strong>allow_single_file_parallelism</strong> – Controls whether DataFusion will attempt to
 speed up writing parquet files by serializing them in parallel. Each
 column in each row group in each output file are serialized in parallel
-leveraging a maximum possible core count of n_files * n_row_groups *
-n_columns.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.maximum_parallel_row_group_writers">
-<span class="sig-name descname"><span class="pre">maximum_parallel_row_group_writers</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.maximum_parallel_row_group_writers" title="Link to this definition">¶</a></dt>
-<dd><p>By default parallel parquet writer is
+leveraging a maximum possible core count of
+<code class="docutils literal notranslate"><span class="pre">n_files</span> <span class="pre">*</span> <span class="pre">n_row_groups</span> <span class="pre">*</span> <span class="pre">n_columns</span></code>.</p></li>
+<li><p><strong>maximum_parallel_row_group_writers</strong> – By default parallel parquet writer is
 tuned for minimum memory usage in a streaming execution plan. You may
 see a performance benefit when writing large parquet files by increasing
-<cite>maximum_parallel_row_group_writers</cite> and
-<cite>maximum_buffered_record_batches_per_stream</cite> if your system has idle
+<code class="docutils literal notranslate"><span class="pre">maximum_parallel_row_group_writers</span></code> and
+<code class="docutils literal notranslate"><span class="pre">maximum_buffered_record_batches_per_stream</span></code> if your system has idle
 cores and can tolerate additional memory usage. Boosting these values is
 likely worthwhile when writing out already in-memory data, such as from
-a cached data frame.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">
-<span class="sig-name descname"><span class="pre">maximum_buffered_record_batches_per_stream</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.maximum_buffered_record_batches_per_stream" title="Link to this definition">¶</a></dt>
-<dd><p>See
-<cite>maximum_parallel_row_group_writers</cite>.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.column_specific_options">
-<span class="sig-name descname"><span class="pre">column_specific_options</span></span><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.column_specific_options" title="Link to this definition">¶</a></dt>
-<dd><p>Overrides options for specific columns. If a column
-is not a part of this dictionary, it will use the parameters provided here.</p>
-</dd></dl>
-
-<p>Initialize the ParquetWriterOptions.</p>
+a cached data frame.</p></li>
+<li><p><strong>maximum_buffered_record_batches_per_stream</strong> – See
+<code class="docutils literal notranslate"><span class="pre">maximum_parallel_row_group_writers</span></code>.</p></li>
+<li><p><strong>column_specific_options</strong> – Overrides options for specific columns. If a column
+is not a part of this dictionary, it will use the parameters provided
+here.</p></li>
+</ul>
+</dd>
+</dl>
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id7">
-<span class="sig-name descname"><span class="pre">allow_single_file_parallelism</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#id7" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.allow_single_file_parallelism">
+<span class="sig-name descname"><span class="pre">allow_single_file_parallelism</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.allow_single_file_parallelism" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id8">
-<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id8" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.bloom_filter_fpp">
+<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_fpp" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id9">
-<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id9" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.bloom_filter_ndv">
+<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_ndv" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id10">
-<span class="sig-name descname"><span class="pre">bloom_filter_on_write</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#id10" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.bloom_filter_on_write">
+<span class="sig-name descname"><span class="pre">bloom_filter_on_write</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.bloom_filter_on_write" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id11">
-<span class="sig-name descname"><span class="pre">column_index_truncate_length</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">64</span></em><a class="headerlink" href="#id11" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.column_index_truncate_length">
+<span class="sig-name descname"><span class="pre">column_index_truncate_length</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">64</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.column_index_truncate_length" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id12">
-<span class="sig-name descname"><span class="pre">column_specific_options</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id12" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.column_specific_options">
+<span class="sig-name descname"><span class="pre">column_specific_options</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.column_specific_options" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id13">
-<span class="sig-name descname"><span class="pre">created_by</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'datafusion-python'</span></em><a class="headerlink" href="#id13" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.created_by">
+<span class="sig-name descname"><span class="pre">created_by</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'datafusion-python'</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.created_by" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id14">
-<span class="sig-name descname"><span class="pre">data_page_row_count_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">20000</span></em><a class="headerlink" href="#id14" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.data_page_row_count_limit">
+<span class="sig-name descname"><span class="pre">data_page_row_count_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">20000</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.data_page_row_count_limit" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id15">
-<span class="sig-name descname"><span class="pre">data_pagesize_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#id15" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.data_pagesize_limit">
+<span class="sig-name descname"><span class="pre">data_pagesize_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.data_pagesize_limit" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id16">
-<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#id16" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.dictionary_enabled">
+<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.dictionary_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id17">
-<span class="sig-name descname"><span class="pre">dictionary_page_size_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#id17" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.dictionary_page_size_limit">
+<span class="sig-name descname"><span class="pre">dictionary_page_size_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.dictionary_page_size_limit" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id18">
-<span class="sig-name descname"><span class="pre">encoding</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id18" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.encoding">
+<span class="sig-name descname"><span class="pre">encoding</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.encoding" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id19">
-<span class="sig-name descname"><span class="pre">max_row_group_size</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#id19" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.max_row_group_size">
+<span class="sig-name descname"><span class="pre">max_row_group_size</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.max_row_group_size" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id20">
-<span class="sig-name descname"><span class="pre">maximum_buffered_record_batches_per_stream</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">2</span></em><a class="headerlink" href="#id20" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">
+<span class="sig-name descname"><span class="pre">maximum_buffered_record_batches_per_stream</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">2</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.maximum_buffered_record_batches_per_stream" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id21">
-<span class="sig-name descname"><span class="pre">maximum_parallel_row_group_writers</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1</span></em><a class="headerlink" href="#id21" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.maximum_parallel_row_group_writers">
+<span class="sig-name descname"><span class="pre">maximum_parallel_row_group_writers</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.maximum_parallel_row_group_writers" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id22">
-<span class="sig-name descname"><span class="pre">skip_arrow_metadata</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#id22" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.skip_arrow_metadata">
+<span class="sig-name descname"><span class="pre">skip_arrow_metadata</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.skip_arrow_metadata" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id23">
-<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'page'</span></em><a class="headerlink" href="#id23" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.statistics_enabled">
+<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'page'</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.statistics_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id24">
-<span class="sig-name descname"><span class="pre">statistics_truncate_length</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id24" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.statistics_truncate_length">
+<span class="sig-name descname"><span class="pre">statistics_truncate_length</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.statistics_truncate_length" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id25">
-<span class="sig-name descname"><span class="pre">write_batch_size</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1024</span></em><a class="headerlink" href="#id25" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.write_batch_size">
+<span class="sig-name descname"><span class="pre">write_batch_size</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1024</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.write_batch_size" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id26">
-<span class="sig-name descname"><span class="pre">writer_version</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'1.0'</span></em><a class="headerlink" href="#id26" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.dataframe.ParquetWriterOptions.writer_version">
+<span class="sig-name descname"><span class="pre">writer_version</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'1.0'</span></em><a class="headerlink" href="#datafusion.dataframe.ParquetWriterOptions.writer_version" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 </dd></dl>
diff --git a/autoapi/datafusion/expr/index.html b/autoapi/datafusion/expr/index.html
index 0b21f068..69206736 100644
--- a/autoapi/datafusion/expr/index.html
+++ b/autoapi/datafusion/expr/index.html
@@ -417,6 +417,11 @@
    Classes
   </a>
  </li>
+ <li class="toc-h2 nav-item toc-entry">
+  <a class="reference internal nav-link" href="#functions">
+   Functions
+  </a>
+ </li>
  <li class="toc-h2 nav-item toc-entry">
   <a class="reference internal nav-link" href="#module-contents">
    Module Contents
@@ -1793,6 +1798,24 @@
      </li>
     </ul>
    </li>
+   <li class="toc-h3 nav-item toc-entry">
+    <a class="reference internal nav-link" href="#datafusion.expr.ensure_expr">
+     <code class="docutils literal notranslate">
+      <span class="pre">
+       ensure_expr()
+      </span>
+     </code>
+    </a>
+   </li>
+   <li class="toc-h3 nav-item toc-entry">
+    <a class="reference internal nav-link" href="#datafusion.expr.ensure_expr_list">
+     <code class="docutils literal notranslate">
+      <span class="pre">
+       ensure_expr_list()
+      </span>
+     </code>
+    </a>
+   </li>
    <li class="toc-h3 nav-item toc-entry">
     <a class="reference internal nav-link" href="#datafusion.expr.Aggregate">
      <code class="docutils literal notranslate">
@@ -2027,6 +2050,15 @@
      </code>
     </a>
    </li>
+   <li class="toc-h3 nav-item toc-entry">
+    <a class="reference internal nav-link" href="#datafusion.expr.EXPR_TYPE_ERROR">
+     <code class="docutils literal notranslate">
+      <span class="pre">
+       EXPR_TYPE_ERROR
+      </span>
+     </code>
+    </a>
+   </li>
    <li class="toc-h3 nav-item toc-entry">
     <a class="reference internal nav-link" href="#datafusion.expr.EmptyRelation">
      <code class="docutils literal notranslate">
@@ -2378,6 +2410,15 @@
      </code>
     </a>
    </li>
+   <li class="toc-h3 nav-item toc-entry">
+    <a class="reference internal nav-link" href="#datafusion.expr.SortKey">
+     <code class="docutils literal notranslate">
+      <span class="pre">
+       SortKey
+      </span>
+     </code>
+    </a>
+   </li>
    <li class="toc-h3 nav-item toc-entry">
     <a class="reference internal nav-link" href="#datafusion.expr.Subquery">
      <code class="docutils literal notranslate">
@@ -2613,121 +2654,127 @@
 <tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.DropView" title="datafusion.expr.DropView"><code class="xref py py-obj docutils literal notranslate"><span class="pre">DropView</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.EmptyRelation" title="datafusion.expr.EmptyRelation"><code class="xref py py-obj docutils literal notranslate"><span class="pre">EmptyRelation</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.EXPR_TYPE_ERROR" title="datafusion.expr.EXPR_TYPE_ERROR"><code class="xref py py-obj docutils literal notranslate"><span class="pre">EXPR_TYPE_ERROR</span></code></a></p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.EmptyRelation" title="datafusion.expr.EmptyRelation"><code class="xref py py-obj docutils literal notranslate"><span class="pre">EmptyRelation</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Execute" title="datafusion.expr.Execute"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Execute</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Execute" title="datafusion.expr.Execute"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Execute</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Exists" title="datafusion.expr.Exists"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Exists</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Exists" title="datafusion.expr.Exists"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Exists</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Explain" title="datafusion.expr.Explain"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Explain</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Explain" title="datafusion.expr.Explain"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Explain</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Extension" title="datafusion.expr.Extension"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Extension</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Extension" title="datafusion.expr.Extension"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Extension</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.FileType" title="datafusion.expr.FileType"><code class="xref py py-obj docutils literal notranslate"><span class="pre">FileType</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.FileType" title="datafusion.expr.FileType"><code class="xref py py-obj docutils literal notranslate"><span class="pre">FileType</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Filter" title="datafusion.expr.Filter"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Filter</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Filter" title="datafusion.expr.Filter"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Filter</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.GroupingSet" title="datafusion.expr.GroupingSet"><code class="xref py py-obj docutils literal notranslate"><span class="pre">GroupingSet</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.GroupingSet" title="datafusion.expr.GroupingSet"><code class="xref py py-obj docutils literal notranslate"><span class="pre">GroupingSet</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.ILike" title="datafusion.expr.ILike"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ILike</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.ILike" title="datafusion.expr.ILike"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ILike</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.InList" title="datafusion.expr.InList"><code class="xref py py-obj docutils literal notranslate"><span class="pre">InList</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.InList" title="datafusion.expr.InList"><code class="xref py py-obj docutils literal notranslate"><span class="pre">InList</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.InSubquery" title="datafusion.expr.InSubquery"><code class="xref py py-obj docutils literal notranslate"><span class="pre">InSubquery</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.InSubquery" title="datafusion.expr.InSubquery"><code class="xref py py-obj docutils literal notranslate"><span class="pre">InSubquery</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.IsFalse" title="datafusion.expr.IsFalse"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsFalse</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.IsFalse" title="datafusion.expr.IsFalse"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsFalse</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.IsNotFalse" title="datafusion.expr.IsNotFalse"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNotFalse</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.IsNotFalse" title="datafusion.expr.IsNotFalse"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNotFalse</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.IsNotNull" title="datafusion.expr.IsNotNull"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNotNull</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.IsNotNull" title="datafusion.expr.IsNotNull"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNotNull</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.IsNotTrue" title="datafusion.expr.IsNotTrue"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNotTrue</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.IsNotTrue" title="datafusion.expr.IsNotTrue"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNotTrue</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.IsNotUnknown" title="datafusion.expr.IsNotUnknown"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNotUnknown</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.IsNotUnknown" title="datafusion.expr.IsNotUnknown"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNotUnknown</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.IsNull" title="datafusion.expr.IsNull"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNull</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.IsNull" title="datafusion.expr.IsNull"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsNull</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.IsTrue" title="datafusion.expr.IsTrue"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsTrue</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.IsTrue" title="datafusion.expr.IsTrue"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsTrue</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.IsUnknown" title="datafusion.expr.IsUnknown"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsUnknown</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.IsUnknown" title="datafusion.expr.IsUnknown"><code class="xref py py-obj docutils literal notranslate"><span class="pre">IsUnknown</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Join" title="datafusion.expr.Join"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Join</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Join" title="datafusion.expr.Join"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Join</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.JoinConstraint" title="datafusion.expr.JoinConstraint"><code class="xref py py-obj docutils literal notranslate"><span class="pre">JoinConstraint</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.JoinConstraint" title="datafusion.expr.JoinConstraint"><code class="xref py py-obj docutils literal notranslate"><span class="pre">JoinConstraint</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.JoinType" title="datafusion.expr.JoinType"><code class="xref py py-obj docutils literal notranslate"><span class="pre">JoinType</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.JoinType" title="datafusion.expr.JoinType"><code class="xref py py-obj docutils literal notranslate"><span class="pre">JoinType</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Like" title="datafusion.expr.Like"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Like</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Like" title="datafusion.expr.Like"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Like</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Limit" title="datafusion.expr.Limit"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Limit</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Limit" title="datafusion.expr.Limit"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Limit</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Literal" title="datafusion.expr.Literal"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Literal</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Literal" title="datafusion.expr.Literal"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Literal</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Negative" title="datafusion.expr.Negative"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Negative</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Negative" title="datafusion.expr.Negative"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Negative</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Not" title="datafusion.expr.Not"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Not</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Not" title="datafusion.expr.Not"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Not</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.OperateFunctionArg" title="datafusion.expr.OperateFunctionArg"><code class="xref py py-obj docutils literal notranslate"><span class="pre">OperateFunctionArg</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.OperateFunctionArg" title="datafusion.expr.OperateFunctionArg"><code class="xref py py-obj docutils literal notranslate"><span class="pre">OperateFunctionArg</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Partitioning" title="datafusion.expr.Partitioning"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Partitioning</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Partitioning" title="datafusion.expr.Partitioning"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Partitioning</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Placeholder" title="datafusion.expr.Placeholder"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Placeholder</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Placeholder" title="datafusion.expr.Placeholder"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Placeholder</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Prepare" title="datafusion.expr.Prepare"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Prepare</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Prepare" title="datafusion.expr.Prepare"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Prepare</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Projection" title="datafusion.expr.Projection"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Projection</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Projection" title="datafusion.expr.Projection"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Projection</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.RecursiveQuery" title="datafusion.expr.RecursiveQuery"><code class="xref py py-obj docutils literal notranslate"><span class="pre">RecursiveQuery</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.RecursiveQuery" title="datafusion.expr.RecursiveQuery"><code class="xref py py-obj docutils literal notranslate"><span class="pre">RecursiveQuery</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Repartition" title="datafusion.expr.Repartition"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Repartition</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Repartition" title="datafusion.expr.Repartition"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Repartition</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.ScalarSubquery" title="datafusion.expr.ScalarSubquery"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ScalarSubquery</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.ScalarSubquery" title="datafusion.expr.ScalarSubquery"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ScalarSubquery</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.ScalarVariable" title="datafusion.expr.ScalarVariable"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ScalarVariable</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.ScalarVariable" title="datafusion.expr.ScalarVariable"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ScalarVariable</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.SetVariable" title="datafusion.expr.SetVariable"><code class="xref py py-obj docutils literal notranslate"><span class="pre">SetVariable</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.SetVariable" title="datafusion.expr.SetVariable"><code class="xref py py-obj docutils literal notranslate"><span class="pre">SetVariable</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.SimilarTo" title="datafusion.expr.SimilarTo"><code class="xref py py-obj docutils literal notranslate"><span class="pre">SimilarTo</span></code></a></p></td>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.SimilarTo" title="datafusion.expr.SimilarTo"><code class="xref py py-obj docutils literal notranslate"><span class="pre">SimilarTo</span></code></a></p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.Sort" title="datafusion.expr.Sort"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Sort</span></code></a></p></td>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Sort" title="datafusion.expr.Sort"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Sort</span></code></a></p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.SortKey" title="datafusion.expr.SortKey"><code class="xref py py-obj docutils literal notranslate"><span class="pre">SortKey</span></code></a></p></td>
 <td><p></p></td>
 </tr>
 <tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.Subquery" title="datafusion.expr.Subquery"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Subquery</span></code></a></p></td>
@@ -2800,6 +2847,19 @@
 </tbody>
 </table>
 </section>
+<section id="functions">
+<h2>Functions<a class="headerlink" href="#functions" title="Link to this heading">¶</a></h2>
+<table class="autosummary longtable table autosummary">
+<tbody>
+<tr class="row-odd"><td><p><a class="reference internal" href="#datafusion.expr.ensure_expr" title="datafusion.expr.ensure_expr"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ensure_expr</span></code></a>(→ datafusion._internal.expr.Expr)</p></td>
+<td><p>Return the internal expression from <code class="docutils literal notranslate"><span class="pre">Expr</span></code> or raise <code class="docutils literal notranslate"><span class="pre">TypeError</span></code>.</p></td>
+</tr>
+<tr class="row-even"><td><p><a class="reference internal" href="#datafusion.expr.ensure_expr_list" title="datafusion.expr.ensure_expr_list"><code class="xref py py-obj docutils literal notranslate"><span class="pre">ensure_expr_list</span></code></a>(→ list[datafusion._internal.expr.Expr])</p></td>
+<td><p>Flatten an iterable of expressions, validating each via <code class="docutils literal notranslate"><span class="pre">ensure_expr</span></code>.</p></td>
+</tr>
+</tbody>
+</table>
+</section>
 <section id="module-contents">
 <h2>Module Contents<a class="headerlink" href="#module-contents" title="Link to this heading">¶</a></h2>
 <dl class="py class">
@@ -2810,7 +2870,7 @@
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">f</span>
 <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">lit</span><span class="p">,</span> <span class="n">col</span>
 <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span>
-    <span class="n">f</span><span class="o">.</span><span class="n">case</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;column_a&quot;</span><span class="p">)</span>
+    <span class="n">f</span><span class="o">.</span><span class="n">case</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;column_a&quot;</span><span class="p">))</span>
     <span class="o">.</span><span class="n">when</span><span class="p">(</span><span class="n">lit</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="n">lit</span><span class="p">(</span><span class="s2">&quot;One&quot;</span><span class="p">))</span>
     <span class="o">.</span><span class="n">when</span><span class="p">(</span><span class="n">lit</span><span class="p">(</span><span class="mi">2</span><span class="p">),</span> <span class="n">lit</span><span class="p">(</span><span class="s2">&quot;Two&quot;</span><span class="p">))</span>
     <span class="o">.</span><span class="n">otherwise</span><span class="p">(</span><span class="n">lit</span><span class="p">(</span><span class="s2">&quot;Unknown&quot;</span><span class="p">))</span>
@@ -2886,7 +2946,12 @@
 <dd><p>Retrieve sub-object.</p>
 <p>If <code class="docutils literal notranslate"><span class="pre">key</span></code> is a string, returns the subfield of the struct.
 If <code class="docutils literal notranslate"><span class="pre">key</span></code> is an integer, retrieves the element in the array. Note that the
-element index begins at <code class="docutils literal notranslate"><span class="pre">0</span></code>, unlike <cite>array_element</cite> which begins at <code class="docutils literal notranslate"><span class="pre">1</span></code>.</p>
+element index begins at <code class="docutils literal notranslate"><span class="pre">0</span></code>, unlike
+<a class="reference internal" href="../functions/index.html#datafusion.functions.array_element" title="datafusion.functions.array_element"><code class="xref py py-func docutils literal notranslate"><span class="pre">array_element()</span></code></a> which begins at <code class="docutils literal notranslate"><span class="pre">1</span></code>.
+If <code class="docutils literal notranslate"><span class="pre">key</span></code> is a slice, returns an array that contains a slice of the
+original array. Similar to integer indexing, this follows Python convention
+where the index begins at <code class="docutils literal notranslate"><span class="pre">0</span></code> unlike
+<a class="reference internal" href="../functions/index.html#datafusion.functions.array_slice" title="datafusion.functions.array_slice"><code class="xref py py-func docutils literal notranslate"><span class="pre">array_slice()</span></code></a> which begins at <code class="docutils literal notranslate"><span class="pre">1</span></code>.</p>
 </dd></dl>
 
 <dl class="py method">
@@ -3137,7 +3202,7 @@ element index begins at <code class="docutils literal notranslate"><span class="
 
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.expr.Expr.cast">
-<span class="sig-name descname"><span class="pre">cast</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">to</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">pyarrow.DataType</span><span class="p"><span class="pre">[</span></span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span><span class="w"> </span><span class="p"><span class="pre">|</spa [...]
+<span class="sig-name descname"><span class="pre">cast</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">to</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">pyarrow.DataType</span><span class="p"><span class="pre">[</span></span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span><span class="w"> </span><span class="p"><span class="pre">|</spa [...]
 <dd><p>Cast to a new data type.</p>
 </dd></dl>
 
@@ -3734,7 +3799,7 @@ type of expression, an error will be generated when <code class="docutils litera
 
 <dl class="py class">
 <dt class="sig sig-object py" id="datafusion.expr.Window">
-<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">Window</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span  [...]
+<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">Window</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span  [...]
 <dd><p>Define reusable window parameters.</p>
 <p>Construct a window definition.</p>
 <dl class="field-list simple">
@@ -3861,6 +3926,43 @@ parameter must be set.</p></li>
 
 </dd></dl>
 
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.expr.ensure_expr">
+<span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">ensure_expr</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">value</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">Expr</span></a><span class= [...]
+<dd><p>Return the internal expression from <code class="docutils literal notranslate"><span class="pre">Expr</span></code> or raise <code class="docutils literal notranslate"><span class="pre">TypeError</span></code>.</p>
+<p>This helper rejects plain strings and other non-<a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><code class="xref py py-class docutils literal notranslate"><span class="pre">Expr</span></code></a> values so
+higher level APIs consistently require explicit <a class="reference internal" href="../index.html#datafusion.col" title="datafusion.col"><code class="xref py py-func docutils literal notranslate"><span class="pre">col()</span></code></a> or
+<a class="reference internal" href="../index.html#datafusion.lit" title="datafusion.lit"><code class="xref py py-func docutils literal notranslate"><span class="pre">lit()</span></code></a> expressions.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>value</strong> – Candidate expression or other object.</p>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>The internal expression representation.</p>
+</dd>
+<dt class="field-odd">Raises<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>TypeError</strong> – If <code class="docutils literal notranslate"><span class="pre">value</span></code> is not an instance of <a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><code class="xref py py-class docutils literal notranslate"><span class="pre">Expr</span></code></a>.</p>
+</dd>
+</dl>
+</dd></dl>
+
+<dl class="py function">
+<dt class="sig sig-object py" id="datafusion.expr.ensure_expr_list">
+<span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">ensure_expr_list</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">exprs</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Iterable</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="#datafusio [...]
+<dd><p>Flatten an iterable of expressions, validating each via <code class="docutils literal notranslate"><span class="pre">ensure_expr</span></code>.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>exprs</strong> – Possibly nested iterable containing expressions.</p>
+</dd>
+<dt class="field-even">Returns<span class="colon">:</span></dt>
+<dd class="field-even"><p>A flat list of raw expressions.</p>
+</dd>
+<dt class="field-odd">Raises<span class="colon">:</span></dt>
+<dd class="field-odd"><p><strong>TypeError</strong> – If any item is not an instance of <a class="reference internal" href="#datafusion.expr.Expr" title="datafusion.expr.Expr"><code class="xref py py-class docutils literal notranslate"><span class="pre">Expr</span></code></a>.</p>
+</dd>
+</dl>
+</dd></dl>
+
 <dl class="py data">
 <dt class="sig sig-object py" id="datafusion.expr.Aggregate">
 <span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">Aggregate</span></span><a class="headerlink" href="#datafusion.expr.Aggregate" title="Link to this definition">¶</a></dt>
@@ -3991,6 +4093,11 @@ parameter must be set.</p></li>
 <span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">DropView</span></span><a class="headerlink" href="#datafusion.expr.DropView" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
+<dl class="py data">
+<dt class="sig sig-object py" id="datafusion.expr.EXPR_TYPE_ERROR">
+<span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">EXPR_TYPE_ERROR</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'Use</span> <span class="pre">col()/column()</span> <span class="pre">or</span> <span class="pre">lit()/literal()</span> <span class="pre">to</span> <span class="pre">construct</span> <span [...]
+<dd></dd></dl>
+
 <dl class="py data">
 <dt class="sig sig-object py" id="datafusion.expr.EmptyRelation">
 <span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">EmptyRelation</span></span><a class="headerlink" href="#datafusion.expr.EmptyRelation" title="Link to this definition">¶</a></dt>
@@ -4186,6 +4293,11 @@ parameter must be set.</p></li>
 <span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">Sort</span></span><a class="headerlink" href="#datafusion.expr.Sort" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
+<dl class="py data">
+<dt class="sig sig-object py" id="datafusion.expr.SortKey">
+<span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">SortKey</span></span><a class="headerlink" href="#datafusion.expr.SortKey" title="Link to this definition">¶</a></dt>
+<dd></dd></dl>
+
 <dl class="py data">
 <dt class="sig sig-object py" id="datafusion.expr.Subquery">
 <span class="sig-prename descclassname"><span class="pre">datafusion.expr.</span></span><span class="sig-name descname"><span class="pre">Subquery</span></span><a class="headerlink" href="#datafusion.expr.Subquery" title="Link to this definition">¶</a></dt>
diff --git a/autoapi/datafusion/functions/index.html b/autoapi/datafusion/functions/index.html
index 88f4b063..c50a7b54 100644
--- a/autoapi/datafusion/functions/index.html
+++ b/autoapi/datafusion/functions/index.html
@@ -3433,7 +3433,7 @@ the options <code class="docutils literal notranslate"><span class="pre">order_b
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.approx_percentile_cont">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">approx_percentile_cont</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span  [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">approx_percentile_cont</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sort_expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr">< [...]
 <dd><p>Returns the value that is approximately at a given percentile of <code class="docutils literal notranslate"><span class="pre">expr</span></code>.</p>
 <p>This aggregate function assumes the input values form a continuous distribution.
 Suppose you have a DataFrame which consists of 100 different test scores. If you
@@ -3441,14 +3441,14 @@ called this function with a percentile of 0.9, it would return the value of the
 test score that is above 90% of the other test scores. The returned value may be
 between two of the values.</p>
 <p>This function uses the [t-digest](<a class="reference external" href="https://arxiv.org/abs/1902.04023">https://arxiv.org/abs/1902.04023</a>) algorithm to
-compute the percentil. You can limit the number of bins used in this algorithm by
+compute the percentile. You can limit the number of bins used in this algorithm by
 setting the <code class="docutils literal notranslate"><span class="pre">num_centroids</span></code> parameter.</p>
 <p>If using the builder functions described in ref:<cite>_aggregation</cite> this function ignores
 the options <code class="docutils literal notranslate"><span class="pre">order_by</span></code>, <code class="docutils literal notranslate"><span class="pre">null_treatment</span></code>, and <code class="docutils literal notranslate"><span class="pre">distinct</span></code>.</p>
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
-<li><p><strong>expression</strong> – Values for which to find the approximate percentile</p></li>
+<li><p><strong>sort_expression</strong> – Values for which to find the approximate percentile</p></li>
 <li><p><strong>percentile</strong> – This must be between 0.0 and 1.0, inclusive</p></li>
 <li><p><strong>num_centroids</strong> – Max bin size for the t-digest algorithm</p></li>
 <li><p><strong>filter</strong> – If provided, only compute against rows for which the filter is True</p></li>
@@ -3459,7 +3459,7 @@ the options <code class="docutils literal notranslate"><span class="pre">order_b
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.approx_percentile_cont_with_weight">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">approx_percentile_cont_with_weight</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr. [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">approx_percentile_cont_with_weight</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">sort_expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion. [...]
 <dd><p>Returns the value of the weighted approximate percentile.</p>
 <p>This aggregate function is similar to <a class="reference internal" href="#datafusion.functions.approx_percentile_cont" title="datafusion.functions.approx_percentile_cont"><code class="xref py py-func docutils literal notranslate"><span class="pre">approx_percentile_cont()</span></code></a> except that
 it uses the associated associated weights.</p>
@@ -3468,9 +3468,10 @@ the options <code class="docutils literal notranslate"><span class="pre">order_b
 <dl class="field-list simple">
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
-<li><p><strong>expression</strong> – Values for which to find the approximate percentile</p></li>
+<li><p><strong>sort_expression</strong> – Values for which to find the approximate percentile</p></li>
 <li><p><strong>weight</strong> – Relative weight for each of the values in <code class="docutils literal notranslate"><span class="pre">expression</span></code></p></li>
 <li><p><strong>percentile</strong> – This must be between 0.0 and 1.0, inclusive</p></li>
+<li><p><strong>num_centroids</strong> – Max bin size for the t-digest algorithm</p></li>
 <li><p><strong>filter</strong> – If provided, only compute against rows for which the filter is True</p></li>
 </ul>
 </dd>
@@ -3486,7 +3487,7 @@ the options <code class="docutils literal notranslate"><span class="pre">order_b
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.array_agg">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">array_agg</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">d [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">array_agg</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">d [...]
 <dd><p>Aggregate values into an array.</p>
 <p>Currently <code class="docutils literal notranslate"><span class="pre">distinct</span></code> and <code class="docutils literal notranslate"><span class="pre">order_by</span></code> cannot be used together. As a work around,
 consider <a class="reference internal" href="#datafusion.functions.array_sort" title="datafusion.functions.array_sort"><code class="xref py py-func docutils literal notranslate"><span class="pre">array_sort()</span></code></a> after aggregation.
@@ -3499,10 +3500,14 @@ the option <code class="docutils literal notranslate"><span class="pre">null_tre
 <li><p><strong>expression</strong> – Values to combine into an array</p></li>
 <li><p><strong>distinct</strong> – If True, a single entry for each distinct value will be in the result</p></li>
 <li><p><strong>filter</strong> – If provided, only compute against rows for which the filter is True</p></li>
-<li><p><strong>order_by</strong> – Order the resultant array values</p></li>
+<li><p><strong>order_by</strong> – Order the resultant array values. Accepts column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">array_agg</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;b&quot;</span><span class="p">))</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -4094,10 +4099,10 @@ the options <code class="docutils literal notranslate"><span class="pre">order_b
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.cume_dist">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">cume_dist</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/i [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">cume_dist</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/i [...]
 <dd><p>Create a cumulative distribution window function.</p>
 <p>This window function is similar to <a class="reference internal" href="#datafusion.functions.rank" title="datafusion.functions.rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">rank()</span></code></a> except that the returned values
-are the ratio of the row number to the total numebr of rows. Here is an example of a
+are the ratio of the row number to the total number of rows. Here is an example of a
 dataframe with a window ordered by descending <code class="docutils literal notranslate"><span class="pre">points</span></code> and the associated
 cumulative distribution:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+-----------+</span>
@@ -4114,10 +4119,15 @@ cumulative distribution:</p>
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
-<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">cume_dist</span><span class="p">(</span><span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;points&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -4178,7 +4188,7 @@ cumulative distribution:</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.dense_rank">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">dense_rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/ [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">dense_rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/ [...]
 <dd><p>Create a dense_rank window function.</p>
 <p>This window function is similar to <a class="reference internal" href="#datafusion.functions.rank" title="datafusion.functions.rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">rank()</span></code></a> except that the returned values
 will be consecutive. Here is an example of a dataframe with a window ordered by
@@ -4197,10 +4207,15 @@ descending <code class="docutils literal notranslate"><span class="pre">points</
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
-<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dense_rank</span><span class="p">(</span><span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;points&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -4259,7 +4274,7 @@ blake2b, and blake3.</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.first_value">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">first_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre" [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">first_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre" [...]
 <dd><p>Returns the first value in a group of values.</p>
 <p>This aggregate function will return the first value in the partition.</p>
 <p>If using the builder functions described in ref:<cite>_aggregation</cite> this function ignores
@@ -4269,11 +4284,16 @@ the option <code class="docutils literal notranslate"><span class="pre">distinct
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>expression</strong> – Argument to perform bitwise calculation on</p></li>
 <li><p><strong>filter</strong> – If provided, only compute against rows for which the filter is True</p></li>
-<li><p><strong>order_by</strong> – Set the ordering of the expression to evaluate</p></li>
+<li><p><strong>order_by</strong> – Set the ordering of the expression to evaluate. Accepts
+column names or expressions.</p></li>
 <li><p><strong>null_treatment</strong> – Assign whether to respect or ignore null values.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">first_value</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;ts&quot;</span><span class="p">))</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -4328,11 +4348,11 @@ characters to lowercase.</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.lag">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">lag</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.exp [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">lag</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.exp [...]
 <dd><p>Create a lag window function.</p>
 <p>Lag operation will return the argument that is in the previous shift_offset-th row
 in the partition. For example <code class="docutils literal notranslate"><span class="pre">lag(col(&quot;b&quot;),</span> <span class="pre">shift_offset=3,</span> <span class="pre">default_value=5)</span></code>
-will return the 3rd previous value in column <code class="docutils literal notranslate"><span class="pre">b</span></code>. At the beginnig of the
+will return the 3rd previous value in column <code class="docutils literal notranslate"><span class="pre">b</span></code>. At the beginning of the
 partition, where no values can be returned it will return the default value of 5.</p>
 <p>Here is an example of both the <code class="docutils literal notranslate"><span class="pre">lag</span></code> and <a class="reference internal" href="#datafusion.functions.lead" title="datafusion.functions.lead"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.lead()</span></code></a>
 functions on a simple DataFrame:</p>
@@ -4353,15 +4373,20 @@ functions on a simple DataFrame:</p>
 <li><p><strong>shift_offset</strong> – Number of rows before the current row.</p></li>
 <li><p><strong>default_value</strong> – Value to return if shift_offet row does not exist.</p></li>
 <li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
-<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">lag</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;b&quot;</span><span class="p">),</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;ts&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.last_value">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">last_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre"> [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">last_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre"> [...]
 <dd><p>Returns the last value in a group of values.</p>
 <p>This aggregate function will return the last value in the partition.</p>
 <p>If using the builder functions described in ref:<cite>_aggregation</cite> this function ignores
@@ -4371,11 +4396,16 @@ the option <code class="docutils literal notranslate"><span class="pre">distinct
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>expression</strong> – Argument to perform bitwise calculation on</p></li>
 <li><p><strong>filter</strong> – If provided, only compute against rows for which the filter is True</p></li>
-<li><p><strong>order_by</strong> – Set the ordering of the expression to evaluate</p></li>
+<li><p><strong>order_by</strong> – Set the ordering of the expression to evaluate. Accepts
+column names or expressions.</p></li>
 <li><p><strong>null_treatment</strong> – Assign whether to respect or ignore null values.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">last_value</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;ts&quot;</span><span class="p">))</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -4386,12 +4416,12 @@ the option <code class="docutils literal notranslate"><span class="pre">distinct
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.lead">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">lead</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.ex [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">lead</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">arg</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">datafusion.ex [...]
 <dd><p>Create a lead window function.</p>
 <p>Lead operation will return the argument that is in the next shift_offset-th row in
 the partition. For example <code class="docutils literal notranslate"><span class="pre">lead(col(&quot;b&quot;),</span> <span class="pre">shift_offset=3,</span> <span class="pre">default_value=5)</span></code> will
 return the 3rd following value in column <code class="docutils literal notranslate"><span class="pre">b</span></code>. At the end of the partition, where
-no futher values can be returned it will return the default value of 5.</p>
+no further values can be returned it will return the default value of 5.</p>
 <p>Here is an example of both the <code class="docutils literal notranslate"><span class="pre">lead</span></code> and <a class="reference internal" href="#datafusion.functions.lag" title="datafusion.functions.lag"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.lag()</span></code></a>
 functions on a simple DataFrame:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+--------+------+-----+</span>
@@ -4413,10 +4443,15 @@ ref:<cite>_window_functions</cite> online documentation.</p>
 <li><p><strong>shift_offset</strong> – Number of rows following the current row.</p></li>
 <li><p><strong>default_value</strong> – Value to return if shift_offet row does not exist.</p></li>
 <li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
-<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">lead</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;b&quot;</span><span class="p">),</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;ts&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -4799,7 +4834,7 @@ the options <code class="docutils literal notranslate"><span class="pre">order_b
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.nth_value">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">nth_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">d [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">nth_value</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre">d [...]
 <dd><p>Returns the n-th value in a group of values.</p>
 <p>This aggregate function will return the n-th value in the partition.</p>
 <p>If using the builder functions described in ref:<cite>_aggregation</cite> this function ignores
@@ -4810,16 +4845,21 @@ the option <code class="docutils literal notranslate"><span class="pre">distinct
 <li><p><strong>expression</strong> – Argument to perform bitwise calculation on</p></li>
 <li><p><strong>n</strong> – Index of value to return. Starts at 1.</p></li>
 <li><p><strong>filter</strong> – If provided, only compute against rows for which the filter is True</p></li>
-<li><p><strong>order_by</strong> – Set the ordering of the expression to evaluate</p></li>
+<li><p><strong>order_by</strong> – Set the ordering of the expression to evaluate. Accepts
+column names or expressions.</p></li>
 <li><p><strong>null_treatment</strong> – Assign whether to respect or ignore null values.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">nth_value</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="mi">2</span><span class="p">,</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;ts&quot;< [...]
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.ntile">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">ntile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">groups</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span cla [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">ntile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">groups</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span cla [...]
 <dd><p>Create a n-tile window function.</p>
 <p>This window function orders the window frame into a give number of groups based on
 the ordering criteria. It then returns which group the current row is assigned to.
@@ -4842,10 +4882,15 @@ and the associated n-tile function:</p>
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>groups</strong> – Number of groups for the n-tile to be divided into.</p></li>
 <li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
-<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ntile</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;points&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -4883,7 +4928,7 @@ extends for <code class="docutils literal notranslate"><span class="pre">length<
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.percent_rank">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">percent_rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../exp [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">percent_rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../exp [...]
 <dd><p>Create a percent_rank window function.</p>
 <p>This window function is similar to <a class="reference internal" href="#datafusion.functions.rank" title="datafusion.functions.rank"><code class="xref py py-func docutils literal notranslate"><span class="pre">rank()</span></code></a> except that the returned values
 are the percentage from 0.0 to 1.0 from first to last. Here is an example of a
@@ -4903,10 +4948,15 @@ rank:</p>
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
-<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">percent_rank</span><span class="p">(</span><span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;points&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -4948,11 +4998,11 @@ rank:</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.rank">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/index. [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">rank</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/index. [...]
 <dd><p>Create a rank window function.</p>
 <p>Returns the rank based upon the window order. Consecutive equal values will receive
 the same rank, but the next different value will not be consecutive but rather the
-number of rows that preceed it plus one. This is similar to Olympic medals. If two
+number of rows that precede it plus one. This is similar to Olympic medals. If two
 people tie for gold, the next place is bronze. There would be no silver medal. Here
 is an example of a dataframe with a window ordered by descending <code class="docutils literal notranslate"><span class="pre">points</span></code> and the
 associated rank.</p>
@@ -4971,10 +5021,15 @@ associated rank.</p>
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
-<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">rank</span><span class="p">(</span><span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;points&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -5217,7 +5272,7 @@ decimal places. You can specify a negative number of decimal places. For example
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.row_number">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">row_number</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/ [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">row_number</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">partition_by</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">list</span><span class="p"><span class="pre">[</span></span><a class="reference internal" href="../expr/ [...]
 <dd><p>Create a row number window function.</p>
 <p>Returns the row number of the window function.</p>
 <p>Here is an example of the <code class="docutils literal notranslate"><span class="pre">row_number</span></code> on a simple DataFrame:</p>
@@ -5235,10 +5290,15 @@ decimal places. You can specify a negative number of decimal places. For example
 <dt class="field-odd">Parameters<span class="colon">:</span></dt>
 <dd class="field-odd"><ul class="simple">
 <li><p><strong>partition_by</strong> – Expressions to partition the window frame on.</p></li>
-<li><p><strong>order_by</strong> – Set ordering within the window frame.</p></li>
+<li><p><strong>order_by</strong> – Set ordering within the window frame. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">row_number</span><span class="p">(</span><span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;points&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -5358,10 +5418,10 @@ the options <code class="docutils literal notranslate"><span class="pre">order_b
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.string_agg">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">string_agg</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre"> [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">string_agg</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">expression</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference internal" href="../expr/index.html#datafusion.expr.Expr" title="datafusion.expr.Expr"><span class="pre"> [...]
 <dd><p>Concatenates the input strings.</p>
 <p>This aggregate function will concatenate input strings, ignoring null values, and
-seperating them with the specified delimiter. Non-string values will be converted to
+separating them with the specified delimiter. Non-string values will be converted to
 their string equivalents.</p>
 <p>If using the builder functions described in ref:<cite>_aggregation</cite> this function ignores
 the options <code class="docutils literal notranslate"><span class="pre">distinct</span></code> and <code class="docutils literal notranslate"><span class="pre">null_treatment</span></code>.</p>
@@ -5371,10 +5431,15 @@ the options <code class="docutils literal notranslate"><span class="pre">distinc
 <li><p><strong>expression</strong> – Argument to perform bitwise calculation on</p></li>
 <li><p><strong>delimiter</strong> – Text to place between each value of expression</p></li>
 <li><p><strong>filter</strong> – If provided, only compute against rows for which the filter is True</p></li>
-<li><p><strong>order_by</strong> – Set the ordering of the expression to evaluate</p></li>
+<li><p><strong>order_by</strong> – Set the ordering of the expression to evaluate. Accepts
+column names or expressions.</p></li>
 </ul>
 </dd>
 </dl>
+<p>For example:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">string_agg</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="s2">&quot;,&quot;</span><span class="p">,</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&q [...]
+</pre></div>
+</div>
 </dd></dl>
 
 <dl class="py function">
@@ -5573,7 +5638,7 @@ detailed usage.</p>
 
 <dl class="py function">
 <dt class="sig sig-object py" id="datafusion.functions.window">
-<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">window</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">args</span></span><span class="p"><s [...]
+<span class="sig-prename descclassname"><span class="pre">datafusion.functions.</span></span><span class="sig-name descname"><span class="pre">window</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">name</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">args</span></span><span class="p"><s [...]
 <dd><p>Creates a new Window function expression.</p>
 <p>This interface will soon be deprecated. Instead of using this interface,
 users should call the window functions directly. For example, to perform a
@@ -5581,6 +5646,10 @@ lag use:</p>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">functions</span><span class="o">.</span><span class="n">lag</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">))</span><span class="o">.</span><span class="n">partition_by</span><span class="p">(</span><span [...]
 </pre></div>
 </div>
+<p>The <code class="docutils literal notranslate"><span class="pre">order_by</span></code> parameter accepts column names or expressions, e.g.:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">window</span><span class="p">(</span><span class="s2">&quot;lag&quot;</span><span class="p">,</span> <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">)],</span> <span class="n">order_by</span><span class="o">=</span><span class="s2">&quot;ts&quot;</span><span class="p">)</span>
+</pre></div>
+</div>
 </dd></dl>
 
 </section>
diff --git a/autoapi/datafusion/index.html b/autoapi/datafusion/index.html
index 390ae840..a6b55070 100644
--- a/autoapi/datafusion/index.html
+++ b/autoapi/datafusion/index.html
@@ -1929,42 +1929,6 @@
      </code>
     </a>
     <ul class="nav section-nav flex-column">
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.encoding">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.encoding
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.dictionary_enabled">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.dictionary_enabled
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.compression">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.compression
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.statistics_enabled">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.statistics_enabled
-        </span>
-       </code>
-      </a>
-     </li>
      <li class="toc-h4 nav-item toc-entry">
       <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.bloom_filter_enabled">
        <code class="docutils literal notranslate">
@@ -1993,34 +1957,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id0">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.bloom_filter_enabled
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id2">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.bloom_filter_fpp
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id3">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetColumnOptions.bloom_filter_ndv
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id4">
+      <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.compression">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetColumnOptions.compression
@@ -2029,7 +1966,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id5">
+      <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.dictionary_enabled">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetColumnOptions.dictionary_enabled
@@ -2038,7 +1975,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id6">
+      <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.encoding">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetColumnOptions.encoding
@@ -2047,7 +1984,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id7">
+      <a class="reference internal nav-link" href="#datafusion.ParquetColumnOptions.statistics_enabled">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetColumnOptions.statistics_enabled
@@ -2066,159 +2003,6 @@
      </code>
     </a>
     <ul class="nav section-nav flex-column">
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.data_pagesize_limit">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.data_pagesize_limit
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.write_batch_size">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.write_batch_size
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.writer_version">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.writer_version
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.skip_arrow_metadata">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.skip_arrow_metadata
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.compression">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.compression
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.dictionary_enabled">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.dictionary_enabled
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.dictionary_page_size_limit">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.dictionary_page_size_limit
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.statistics_enabled">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.statistics_enabled
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.max_row_group_size">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.max_row_group_size
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.created_by">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.created_by
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.column_index_truncate_length">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.column_index_truncate_length
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.statistics_truncate_length">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.statistics_truncate_length
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.data_page_row_count_limit">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.data_page_row_count_limit
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.encoding">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.encoding
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.bloom_filter_on_write">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.bloom_filter_on_write
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.bloom_filter_fpp">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.bloom_filter_fpp
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.bloom_filter_ndv">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.bloom_filter_ndv
-        </span>
-       </code>
-      </a>
-     </li>
      <li class="toc-h4 nav-item toc-entry">
       <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.allow_single_file_parallelism">
        <code class="docutils literal notranslate">
@@ -2229,43 +2013,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.maximum_parallel_row_group_writers">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.maximum_parallel_row_group_writers
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.maximum_buffered_record_batches_per_stream
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.column_specific_options">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.column_specific_options
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id8">
-       <code class="docutils literal notranslate">
-        <span class="pre">
-         ParquetWriterOptions.allow_single_file_parallelism
-        </span>
-       </code>
-      </a>
-     </li>
-     <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id9">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.bloom_filter_fpp">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.bloom_filter_fpp
@@ -2274,7 +2022,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id10">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.bloom_filter_ndv">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.bloom_filter_ndv
@@ -2283,7 +2031,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id11">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.bloom_filter_on_write">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.bloom_filter_on_write
@@ -2292,7 +2040,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id12">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.column_index_truncate_length">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.column_index_truncate_length
@@ -2301,7 +2049,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id13">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.column_specific_options">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.column_specific_options
@@ -2310,7 +2058,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id14">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.created_by">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.created_by
@@ -2319,7 +2067,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id15">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.data_page_row_count_limit">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.data_page_row_count_limit
@@ -2328,7 +2076,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id16">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.data_pagesize_limit">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.data_pagesize_limit
@@ -2337,7 +2085,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id17">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.dictionary_enabled">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.dictionary_enabled
@@ -2346,7 +2094,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id18">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.dictionary_page_size_limit">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.dictionary_page_size_limit
@@ -2355,7 +2103,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id19">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.encoding">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.encoding
@@ -2364,7 +2112,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id20">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.max_row_group_size">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.max_row_group_size
@@ -2373,7 +2121,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id21">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.maximum_buffered_record_batches_per_stream
@@ -2382,7 +2130,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id22">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.maximum_parallel_row_group_writers">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.maximum_parallel_row_group_writers
@@ -2391,7 +2139,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id23">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.skip_arrow_metadata">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.skip_arrow_metadata
@@ -2400,7 +2148,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id24">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.statistics_enabled">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.statistics_enabled
@@ -2409,7 +2157,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id25">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.statistics_truncate_length">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.statistics_truncate_length
@@ -2418,7 +2166,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id26">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.write_batch_size">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.write_batch_size
@@ -2427,7 +2175,7 @@
       </a>
      </li>
      <li class="toc-h4 nav-item toc-entry">
-      <a class="reference internal nav-link" href="#id27">
+      <a class="reference internal nav-link" href="#datafusion.ParquetWriterOptions.writer_version">
        <code class="docutils literal notranslate">
         <span class="pre">
          ParquetWriterOptions.writer_version
@@ -3736,7 +3484,12 @@ single value for unary nodes, or two values for binary nodes (such as joins).</p
 <dd><p>Retrieve sub-object.</p>
 <p>If <code class="docutils literal notranslate"><span class="pre">key</span></code> is a string, returns the subfield of the struct.
 If <code class="docutils literal notranslate"><span class="pre">key</span></code> is an integer, retrieves the element in the array. Note that the
-element index begins at <code class="docutils literal notranslate"><span class="pre">0</span></code>, unlike <cite>array_element</cite> which begins at <code class="docutils literal notranslate"><span class="pre">1</span></code>.</p>
+element index begins at <code class="docutils literal notranslate"><span class="pre">0</span></code>, unlike
+<a class="reference internal" href="functions/index.html#datafusion.functions.array_element" title="datafusion.functions.array_element"><code class="xref py py-func docutils literal notranslate"><span class="pre">array_element()</span></code></a> which begins at <code class="docutils literal notranslate"><span class="pre">1</span></code>.
+If <code class="docutils literal notranslate"><span class="pre">key</span></code> is a slice, returns an array that contains a slice of the
+original array. Similar to integer indexing, this follows Python convention
+where the index begins at <code class="docutils literal notranslate"><span class="pre">0</span></code> unlike
+<a class="reference internal" href="functions/index.html#datafusion.functions.array_slice" title="datafusion.functions.array_slice"><code class="xref py py-func docutils literal notranslate"><span class="pre">array_slice()</span></code></a> which begins at <code class="docutils literal notranslate"><span class="pre">1</span></code>.</p>
 </dd></dl>
 
 <dl class="py method">
@@ -3987,7 +3740,7 @@ element index begins at <code class="docutils literal notranslate"><span class="
 
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.Expr.cast">
-<span class="sig-name descname"><span class="pre">cast</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">to</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">pyarrow.DataType</span><span class="p"><span class="pre">[</span></span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span><span class="w"> </span><span class="p"><span class="pre">|</spa [...]
+<span class="sig-name descname"><span class="pre">cast</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">to</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">pyarrow.DataType</span><span class="p"><span class="pre">[</span></span><span class="pre">Any</span><span class="p"><span class="pre">]</span></span><span class="w"> </span><span class="p"><span class="pre">|</spa [...]
 <dd><p>Cast to a new data type.</p>
 </dd></dl>
 
@@ -4629,96 +4382,68 @@ using the `DOT</cite> language. This format can be visualized using software fro
 <em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.</span></span><span class="sig-name descname"><span class="pre">ParquetColumnOptions</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">encoding</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span><span class="w"> < [...]
 <dd><p>Parquet options for individual columns.</p>
 <p>Contains the available options that can be applied for an individual Parquet column,
-replacing the global options in <cite>ParquetWriterOptions</cite>.</p>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.encoding">
-<span class="sig-name descname"><span class="pre">encoding</span></span><a class="headerlink" href="#datafusion.ParquetColumnOptions.encoding" title="Link to this definition">¶</a></dt>
-<dd><p>Sets encoding for the column path. Valid values are: <cite>plain</cite>,
-<cite>plain_dictionary</cite>, <cite>rle</cite>, <cite>bit_packed</cite>, <cite>delta_binary_packed</cite>,
-<cite>delta_length_byte_array</cite>, <cite>delta_byte_array</cite>, <cite>rle_dictionary</cite>, and
-<cite>byte_stream_split</cite>. These values are not case-sensitive. If <cite>None</cite>, uses
-the default parquet options</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.dictionary_enabled">
-<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><a class="headerlink" href="#datafusion.ParquetColumnOptions.dictionary_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if dictionary encoding is enabled for the column path.
-If <cite>None</cite>, uses the default parquet options</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.compression">
-<span class="sig-name descname"><span class="pre">compression</span></span><a class="headerlink" href="#datafusion.ParquetColumnOptions.compression" title="Link to this definition">¶</a></dt>
-<dd><p>Sets default parquet compression codec for the column path. Valid
-values are <cite>uncompressed</cite>, <cite>snappy</cite>, <cite>gzip(level)</cite>, <cite>lzo</cite>, <cite>brotli(level)</cite>,
-<cite>lz4</cite>, <cite>zstd(level)</cite>, and <cite>lz4_raw</cite>. These values are not case-sensitive. If
-<cite>None</cite>, uses the default parquet options.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.statistics_enabled">
-<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><a class="headerlink" href="#datafusion.ParquetColumnOptions.statistics_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if statistics are enabled for the column Valid values
-are: <cite>none</cite>, <cite>chunk</cite>, and <cite>page</cite> These values are not case sensitive. If
-<cite>None</cite>, uses the default parquet options.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.bloom_filter_enabled">
-<span class="sig-name descname"><span class="pre">bloom_filter_enabled</span></span><a class="headerlink" href="#datafusion.ParquetColumnOptions.bloom_filter_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if bloom filter is enabled for the column path. If
-<cite>None</cite>, uses the default parquet options.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.bloom_filter_fpp">
-<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><a class="headerlink" href="#datafusion.ParquetColumnOptions.bloom_filter_fpp" title="Link to this definition">¶</a></dt>
-<dd><p>Sets bloom filter false positive probability for the column
-path. If <cite>None</cite>, uses the default parquet options.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.bloom_filter_ndv">
-<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><a class="headerlink" href="#datafusion.ParquetColumnOptions.bloom_filter_ndv" title="Link to this definition">¶</a></dt>
-<dd><p>Sets bloom filter number of distinct values. If <cite>None</cite>, uses
-the default parquet options.</p>
-</dd></dl>
-
+replacing the global options in <code class="docutils literal notranslate"><span class="pre">ParquetWriterOptions</span></code>.</p>
 <p>Initialize the ParquetColumnOptions.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>encoding</strong> – Sets encoding for the column path. Valid values are: <code class="docutils literal notranslate"><span class="pre">plain</span></code>,
+<code class="docutils literal notranslate"><span class="pre">plain_dictionary</span></code>, <code class="docutils literal notranslate"><span class="pre">rle</span></code>, <code class="docutils literal notranslate"><span class="pre">bit_packed</span></code>, <code class="docutils literal notranslate"><span class="pre">delta_binary_packed</span></code>,
+<code class="docutils literal notranslate"><span class="pre">delta_length_byte_array</span></code>, <code class="docutils literal notranslate"><span class="pre">delta_byte_array</span></code>, <code class="docutils literal notranslate"><span class="pre">rle_dictionary</span></code>,
+and <code class="docutils literal notranslate"><span class="pre">byte_stream_split</span></code>. These values are not case-sensitive. If
+<code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet options</p></li>
+<li><p><strong>dictionary_enabled</strong> – Sets if dictionary encoding is enabled for the column
+path. If <cite>None</cite>, uses the default parquet options</p></li>
+<li><p><strong>compression</strong> – Sets default parquet compression codec for the column path.
+Valid values are <code class="docutils literal notranslate"><span class="pre">uncompressed</span></code>, <code class="docutils literal notranslate"><span class="pre">snappy</span></code>, <code class="docutils literal notranslate"><span class="pre">gzip(level)</span></code>, <code class="docutils literal notranslate"><span class="pre">lzo</span></code>,
+<code class="docutils literal notranslate"><span class="pre">brotli(level)</span></code>, <code class="docutils literal notranslate"><span class="pre">lz4</span></code>, <code class="docutils literal notranslate"><span class="pre">zstd(level)</span></code>, and <code class="docutils literal notranslate"><span class="pre">lz4_raw</span></code>. These
+values are not case-sensitive. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet
+options.</p></li>
+<li><p><strong>statistics_enabled</strong> – Sets if statistics are enabled for the column Valid
+values are: <code class="docutils literal notranslate"><span class="pre">none</span></code>, <code class="docutils literal notranslate"><span class="pre">chunk</span></code>, and <code class="docutils literal notranslate"><span class="pre">page</span></code> These values are not case
+sensitive. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet options.</p></li>
+<li><p><strong>bloom_filter_enabled</strong> – Sets if bloom filter is enabled for the column path.
+If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet options.</p></li>
+<li><p><strong>bloom_filter_fpp</strong> – Sets bloom filter false positive probability for the
+column path. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet options.</p></li>
+<li><p><strong>bloom_filter_ndv</strong> – Sets bloom filter number of distinct values. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet options.</p></li>
+</ul>
+</dd>
+</dl>
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id0">
-<span class="sig-name descname"><span class="pre">bloom_filter_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id0" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.bloom_filter_enabled">
+<span class="sig-name descname"><span class="pre">bloom_filter_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetColumnOptions.bloom_filter_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id2">
-<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id2" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.bloom_filter_fpp">
+<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetColumnOptions.bloom_filter_fpp" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id3">
-<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id3" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.bloom_filter_ndv">
+<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetColumnOptions.bloom_filter_ndv" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id4">
-<span class="sig-name descname"><span class="pre">compression</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id4" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.compression">
+<span class="sig-name descname"><span class="pre">compression</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetColumnOptions.compression" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id5">
-<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id5" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.dictionary_enabled">
+<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetColumnOptions.dictionary_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id6">
-<span class="sig-name descname"><span class="pre">encoding</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id6" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.encoding">
+<span class="sig-name descname"><span class="pre">encoding</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetColumnOptions.encoding" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id7">
-<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id7" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetColumnOptions.statistics_enabled">
+<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetColumnOptions.statistics_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 </dd></dl>
@@ -4728,270 +4453,177 @@ the default parquet options.</p>
 <em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">datafusion.</span></span><span class="sig-name descname"><span class="pre">ParquetWriterOptions</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">data_pagesize_limit</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span> [...]
 <dd><p>Advanced parquet writer options.</p>
 <p>Allows settings the writer options that apply to the entire file. Some options can
-also be set on a column by column basis, with the field <cite>column_specific_options</cite>
-(see <cite>ParquetColumnOptions</cite>).</p>
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.data_pagesize_limit">
-<span class="sig-name descname"><span class="pre">data_pagesize_limit</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.data_pagesize_limit" title="Link to this definition">¶</a></dt>
-<dd><p>Sets best effort maximum size of data page in bytes.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.write_batch_size">
-<span class="sig-name descname"><span class="pre">write_batch_size</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.write_batch_size" title="Link to this definition">¶</a></dt>
-<dd><p>Sets write_batch_size in bytes.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.writer_version">
-<span class="sig-name descname"><span class="pre">writer_version</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.writer_version" title="Link to this definition">¶</a></dt>
-<dd><p>Sets parquet writer version. Valid values are <cite>1.0</cite> and
-<cite>2.0</cite>.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.skip_arrow_metadata">
-<span class="sig-name descname"><span class="pre">skip_arrow_metadata</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.skip_arrow_metadata" title="Link to this definition">¶</a></dt>
-<dd><p>Skip encoding the embedded arrow metadata in the
-KV_meta.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.compression">
-<span class="sig-name descname"><span class="pre">compression</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.compression" title="Link to this definition">¶</a></dt>
-<dd><p>Compression type to use. Default is “zstd(3)”.
-Available compression types are
-- “uncompressed”: No compression.
-- “snappy”: Snappy compression.
-- “gzip(n)”: Gzip compression with level n.
-- “brotli(n)”: Brotli compression with level n.
-- “lz4”: LZ4 compression.
-- “lz4_raw”: LZ4_RAW compression.
-- “zstd(n)”: Zstandard compression with level n.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.dictionary_enabled">
-<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.dictionary_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if dictionary encoding is enabled. If None, uses
-the default parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.dictionary_page_size_limit">
-<span class="sig-name descname"><span class="pre">dictionary_page_size_limit</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.dictionary_page_size_limit" title="Link to this definition">¶</a></dt>
-<dd><p>Sets best effort maximum dictionary page size,
-in bytes.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.statistics_enabled">
-<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.statistics_enabled" title="Link to this definition">¶</a></dt>
-<dd><p>Sets if statistics are enabled for any column Valid
-values are <cite>none</cite>, <cite>chunk</cite>, and <cite>page</cite>. If None, uses the default
-parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.max_row_group_size">
-<span class="sig-name descname"><span class="pre">max_row_group_size</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.max_row_group_size" title="Link to this definition">¶</a></dt>
-<dd><p>Target maximum number of rows in each row group
-(defaults to 1M rows). Writing larger row groups requires more memory to
-write, but can get better compression and be faster to read.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.created_by">
-<span class="sig-name descname"><span class="pre">created_by</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.created_by" title="Link to this definition">¶</a></dt>
-<dd><p>Sets “created by” property.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.column_index_truncate_length">
-<span class="sig-name descname"><span class="pre">column_index_truncate_length</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.column_index_truncate_length" title="Link to this definition">¶</a></dt>
-<dd><p>Sets column index truncate length.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.statistics_truncate_length">
-<span class="sig-name descname"><span class="pre">statistics_truncate_length</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.statistics_truncate_length" title="Link to this definition">¶</a></dt>
-<dd><p>Sets statistics truncate length. If None, uses
-the default parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.data_page_row_count_limit">
-<span class="sig-name descname"><span class="pre">data_page_row_count_limit</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.data_page_row_count_limit" title="Link to this definition">¶</a></dt>
-<dd><p>Sets best effort maximum number of rows in a data
-page.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.encoding">
-<span class="sig-name descname"><span class="pre">encoding</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.encoding" title="Link to this definition">¶</a></dt>
-<dd><p>Sets default encoding for any column. Valid values are <cite>plain</cite>,
-<cite>plain_dictionary</cite>, <cite>rle</cite>, <cite>bit_packed</cite>, <cite>delta_binary_packed</cite>,
-<cite>delta_length_byte_array</cite>, <cite>delta_byte_array</cite>, <cite>rle_dictionary</cite>, and
-<cite>byte_stream_split</cite>. If None, uses the default parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.bloom_filter_on_write">
-<span class="sig-name descname"><span class="pre">bloom_filter_on_write</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.bloom_filter_on_write" title="Link to this definition">¶</a></dt>
-<dd><p>Write bloom filters for all columns when creating
-parquet files.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.bloom_filter_fpp">
-<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.bloom_filter_fpp" title="Link to this definition">¶</a></dt>
-<dd><p>Sets bloom filter false positive probability. If None,
-uses the default parquet writer setting</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.bloom_filter_ndv">
-<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.bloom_filter_ndv" title="Link to this definition">¶</a></dt>
-<dd><p>Sets bloom filter number of distinct values. If None, uses
-the default parquet writer setting.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.allow_single_file_parallelism">
-<span class="sig-name descname"><span class="pre">allow_single_file_parallelism</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.allow_single_file_parallelism" title="Link to this definition">¶</a></dt>
-<dd><p>Controls whether DataFusion will attempt to
+also be set on a column by column basis, with the field <code class="docutils literal notranslate"><span class="pre">column_specific_options</span></code>
+(see <code class="docutils literal notranslate"><span class="pre">ParquetColumnOptions</span></code>).</p>
+<p>Initialize the ParquetWriterOptions.</p>
+<dl class="field-list simple">
+<dt class="field-odd">Parameters<span class="colon">:</span></dt>
+<dd class="field-odd"><ul class="simple">
+<li><p><strong>data_pagesize_limit</strong> – Sets best effort maximum size of data page in bytes.</p></li>
+<li><p><strong>write_batch_size</strong> – Sets write_batch_size in bytes.</p></li>
+<li><p><strong>writer_version</strong> – Sets parquet writer version. Valid values are <code class="docutils literal notranslate"><span class="pre">1.0</span></code> and
+<code class="docutils literal notranslate"><span class="pre">2.0</span></code>.</p></li>
+<li><p><strong>skip_arrow_metadata</strong> – Skip encoding the embedded arrow metadata in the
+KV_meta.</p></li>
+<li><p><strong>compression</strong> – <p>Compression type to use. Default is <code class="docutils literal notranslate"><span class="pre">zstd(3)</span></code>.
+Available compression types are</p>
+<ul>
+<li><p><code class="docutils literal notranslate"><span class="pre">uncompressed</span></code>: No compression.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">snappy</span></code>: Snappy compression.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">gzip(n)</span></code>: Gzip compression with level n.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">brotli(n)</span></code>: Brotli compression with level n.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">lz4</span></code>: LZ4 compression.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">lz4_raw</span></code>: LZ4_RAW compression.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">zstd(n)</span></code>: Zstandard compression with level n.</p></li>
+</ul>
+</p></li>
+<li><p><strong>compression_level</strong> – Compression level to set.</p></li>
+<li><p><strong>dictionary_enabled</strong> – Sets if dictionary encoding is enabled. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet writer setting.</p></li>
+<li><p><strong>dictionary_page_size_limit</strong> – Sets best effort maximum dictionary page size,
+in bytes.</p></li>
+<li><p><strong>statistics_enabled</strong> – Sets if statistics are enabled for any column Valid
+values are <code class="docutils literal notranslate"><span class="pre">none</span></code>, <code class="docutils literal notranslate"><span class="pre">chunk</span></code>, and <code class="docutils literal notranslate"><span class="pre">page</span></code>. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the
+default parquet writer setting.</p></li>
+<li><p><strong>max_row_group_size</strong> – Target maximum number of rows in each row group
+(defaults to 1M rows). Writing larger row groups requires more memory
+to write, but can get better compression and be faster to read.</p></li>
+<li><p><strong>created_by</strong> – Sets “created by” property.</p></li>
+<li><p><strong>column_index_truncate_length</strong> – Sets column index truncate length.</p></li>
+<li><p><strong>statistics_truncate_length</strong> – Sets statistics truncate length. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet writer setting.</p></li>
+<li><p><strong>data_page_row_count_limit</strong> – Sets best effort maximum number of rows in a data
+page.</p></li>
+<li><p><strong>encoding</strong> – Sets default encoding for any column. Valid values are <code class="docutils literal notranslate"><span class="pre">plain</span></code>,
+<code class="docutils literal notranslate"><span class="pre">plain_dictionary</span></code>, <code class="docutils literal notranslate"><span class="pre">rle</span></code>, <code class="docutils literal notranslate"><span class="pre">bit_packed</span></code>, <code class="docutils literal notranslate"><span class="pre">delta_binary_packed</span></code>,
+<code class="docutils literal notranslate"><span class="pre">delta_length_byte_array</span></code>, <code class="docutils literal notranslate"><span class="pre">delta_byte_array</span></code>, <code class="docutils literal notranslate"><span class="pre">rle_dictionary</span></code>,
+and <code class="docutils literal notranslate"><span class="pre">byte_stream_split</span></code>. If <code class="docutils literal notranslate"><span class="pre">None</span></code>, uses the default parquet writer
+setting.</p></li>
+<li><p><strong>bloom_filter_on_write</strong> – Write bloom filters for all columns when creating
+parquet files.</p></li>
+<li><p><strong>bloom_filter_fpp</strong> – Sets bloom filter false positive probability. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet writer setting</p></li>
+<li><p><strong>bloom_filter_ndv</strong> – Sets bloom filter number of distinct values. If <code class="docutils literal notranslate"><span class="pre">None</span></code>,
+uses the default parquet writer setting.</p></li>
+<li><p><strong>allow_single_file_parallelism</strong> – Controls whether DataFusion will attempt to
 speed up writing parquet files by serializing them in parallel. Each
 column in each row group in each output file are serialized in parallel
-leveraging a maximum possible core count of n_files * n_row_groups *
-n_columns.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.maximum_parallel_row_group_writers">
-<span class="sig-name descname"><span class="pre">maximum_parallel_row_group_writers</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.maximum_parallel_row_group_writers" title="Link to this definition">¶</a></dt>
-<dd><p>By default parallel parquet writer is
+leveraging a maximum possible core count of
+<code class="docutils literal notranslate"><span class="pre">n_files</span> <span class="pre">*</span> <span class="pre">n_row_groups</span> <span class="pre">*</span> <span class="pre">n_columns</span></code>.</p></li>
+<li><p><strong>maximum_parallel_row_group_writers</strong> – By default parallel parquet writer is
 tuned for minimum memory usage in a streaming execution plan. You may
 see a performance benefit when writing large parquet files by increasing
-<cite>maximum_parallel_row_group_writers</cite> and
-<cite>maximum_buffered_record_batches_per_stream</cite> if your system has idle
+<code class="docutils literal notranslate"><span class="pre">maximum_parallel_row_group_writers</span></code> and
+<code class="docutils literal notranslate"><span class="pre">maximum_buffered_record_batches_per_stream</span></code> if your system has idle
 cores and can tolerate additional memory usage. Boosting these values is
 likely worthwhile when writing out already in-memory data, such as from
-a cached data frame.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">
-<span class="sig-name descname"><span class="pre">maximum_buffered_record_batches_per_stream</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.maximum_buffered_record_batches_per_stream" title="Link to this definition">¶</a></dt>
-<dd><p>See
-<cite>maximum_parallel_row_group_writers</cite>.</p>
-</dd></dl>
-
-<dl class="py attribute">
-<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.column_specific_options">
-<span class="sig-name descname"><span class="pre">column_specific_options</span></span><a class="headerlink" href="#datafusion.ParquetWriterOptions.column_specific_options" title="Link to this definition">¶</a></dt>
-<dd><p>Overrides options for specific columns. If a column
-is not a part of this dictionary, it will use the parameters provided here.</p>
-</dd></dl>
-
-<p>Initialize the ParquetWriterOptions.</p>
+a cached data frame.</p></li>
+<li><p><strong>maximum_buffered_record_batches_per_stream</strong> – See
+<code class="docutils literal notranslate"><span class="pre">maximum_parallel_row_group_writers</span></code>.</p></li>
+<li><p><strong>column_specific_options</strong> – Overrides options for specific columns. If a column
+is not a part of this dictionary, it will use the parameters provided
+here.</p></li>
+</ul>
+</dd>
+</dl>
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id8">
-<span class="sig-name descname"><span class="pre">allow_single_file_parallelism</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#id8" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.allow_single_file_parallelism">
+<span class="sig-name descname"><span class="pre">allow_single_file_parallelism</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.allow_single_file_parallelism" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id9">
-<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id9" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.bloom_filter_fpp">
+<span class="sig-name descname"><span class="pre">bloom_filter_fpp</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.bloom_filter_fpp" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id10">
-<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id10" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.bloom_filter_ndv">
+<span class="sig-name descname"><span class="pre">bloom_filter_ndv</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.bloom_filter_ndv" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id11">
-<span class="sig-name descname"><span class="pre">bloom_filter_on_write</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#id11" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.bloom_filter_on_write">
+<span class="sig-name descname"><span class="pre">bloom_filter_on_write</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.bloom_filter_on_write" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id12">
-<span class="sig-name descname"><span class="pre">column_index_truncate_length</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">64</span></em><a class="headerlink" href="#id12" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.column_index_truncate_length">
+<span class="sig-name descname"><span class="pre">column_index_truncate_length</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">64</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.column_index_truncate_length" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id13">
-<span class="sig-name descname"><span class="pre">column_specific_options</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id13" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.column_specific_options">
+<span class="sig-name descname"><span class="pre">column_specific_options</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.column_specific_options" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id14">
-<span class="sig-name descname"><span class="pre">created_by</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'datafusion-python'</span></em><a class="headerlink" href="#id14" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.created_by">
+<span class="sig-name descname"><span class="pre">created_by</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'datafusion-python'</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.created_by" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id15">
-<span class="sig-name descname"><span class="pre">data_page_row_count_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">20000</span></em><a class="headerlink" href="#id15" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.data_page_row_count_limit">
+<span class="sig-name descname"><span class="pre">data_page_row_count_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">20000</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.data_page_row_count_limit" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id16">
-<span class="sig-name descname"><span class="pre">data_pagesize_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#id16" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.data_pagesize_limit">
+<span class="sig-name descname"><span class="pre">data_pagesize_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.data_pagesize_limit" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id17">
-<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#id17" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.dictionary_enabled">
+<span class="sig-name descname"><span class="pre">dictionary_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">True</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.dictionary_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id18">
-<span class="sig-name descname"><span class="pre">dictionary_page_size_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#id18" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.dictionary_page_size_limit">
+<span class="sig-name descname"><span class="pre">dictionary_page_size_limit</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.dictionary_page_size_limit" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id19">
-<span class="sig-name descname"><span class="pre">encoding</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id19" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.encoding">
+<span class="sig-name descname"><span class="pre">encoding</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.encoding" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id20">
-<span class="sig-name descname"><span class="pre">max_row_group_size</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#id20" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.max_row_group_size">
+<span class="sig-name descname"><span class="pre">max_row_group_size</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1048576</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.max_row_group_size" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id21">
-<span class="sig-name descname"><span class="pre">maximum_buffered_record_batches_per_stream</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">2</span></em><a class="headerlink" href="#id21" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">
+<span class="sig-name descname"><span class="pre">maximum_buffered_record_batches_per_stream</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">2</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.maximum_buffered_record_batches_per_stream" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id22">
-<span class="sig-name descname"><span class="pre">maximum_parallel_row_group_writers</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1</span></em><a class="headerlink" href="#id22" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.maximum_parallel_row_group_writers">
+<span class="sig-name descname"><span class="pre">maximum_parallel_row_group_writers</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.maximum_parallel_row_group_writers" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id23">
-<span class="sig-name descname"><span class="pre">skip_arrow_metadata</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#id23" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.skip_arrow_metadata">
+<span class="sig-name descname"><span class="pre">skip_arrow_metadata</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">False</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.skip_arrow_metadata" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id24">
-<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'page'</span></em><a class="headerlink" href="#id24" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.statistics_enabled">
+<span class="sig-name descname"><span class="pre">statistics_enabled</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'page'</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.statistics_enabled" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id25">
-<span class="sig-name descname"><span class="pre">statistics_truncate_length</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#id25" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.statistics_truncate_length">
+<span class="sig-name descname"><span class="pre">statistics_truncate_length</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">None</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.statistics_truncate_length" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id26">
-<span class="sig-name descname"><span class="pre">write_batch_size</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1024</span></em><a class="headerlink" href="#id26" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.write_batch_size">
+<span class="sig-name descname"><span class="pre">write_batch_size</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">1024</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.write_batch_size" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 <dl class="py attribute">
-<dt class="sig sig-object py" id="id27">
-<span class="sig-name descname"><span class="pre">writer_version</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'1.0'</span></em><a class="headerlink" href="#id27" title="Link to this definition">¶</a></dt>
+<dt class="sig sig-object py" id="datafusion.ParquetWriterOptions.writer_version">
+<span class="sig-name descname"><span class="pre">writer_version</span></span><em class="property"><span class="w"> </span><span class="p"><span class="pre">=</span></span><span class="w"> </span><span class="pre">'1.0'</span></em><a class="headerlink" href="#datafusion.ParquetWriterOptions.writer_version" title="Link to this definition">¶</a></dt>
 <dd></dd></dl>
 
 </dd></dl>
diff --git a/autoapi/datafusion/user_defined/index.html b/autoapi/datafusion/user_defined/index.html
index f9c314e5..c2e1f6cd 100644
--- a/autoapi/datafusion/user_defined/index.html
+++ b/autoapi/datafusion/user_defined/index.html
@@ -1548,7 +1548,7 @@ D | 4
 <dl class="py method">
 <dt class="sig sig-object py" id="datafusion.user_defined.WindowEvaluator.get_range">
 <span class="sig-name descname"><span class="pre">get_range</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">idx</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">num_rows</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre [...]
-<dd><p>Return the range for the window fuction.</p>
+<dd><p>Return the range for the window function.</p>
 <p>If <cite>uses_window_frame</cite> flag is <cite>false</cite>. This method is used to
 calculate required range for the window function during
 stateful execution.</p>
diff --git a/contributor-guide/ffi.html b/contributor-guide/ffi.html
index 518ae0ef..0439f323 100644
--- a/contributor-guide/ffi.html
+++ b/contributor-guide/ffi.html
@@ -608,7 +608,7 @@ optimization levels. If you wish to go down this route, there are two approaches
 have identified you can use.</p>
 <ol class="arabic simple">
 <li><p>Re-export all of <code class="docutils literal notranslate"><span class="pre">datafusion-python</span></code> yourself with your extensions built in.</p></li>
-<li><p>Carefully synchonize your software releases with the <code class="docutils literal notranslate"><span class="pre">datafusion-python</span></code> CI build
+<li><p>Carefully synchronize your software releases with the <code class="docutils literal notranslate"><span class="pre">datafusion-python</span></code> CI build
 system so that your libraries use the exact same compiler, features, and
 optimization level.</p></li>
 </ol>
diff --git a/contributor-guide/introduction.html b/contributor-guide/introduction.html
index 5df6002c..37db3008 100644
--- a/contributor-guide/introduction.html
+++ b/contributor-guide/introduction.html
@@ -474,7 +474,7 @@ and for <cite>uv run</cite> commands the additional parameter <cite>–no-projec
 <p>Bootstrap:</p>
 <div class="highlight-shell notranslate"><div class="highlight"><pre><span></span><span class="c1"># fetch this repo</span>
 git<span class="w"> </span>clone<span class="w"> </span>git@github.com:apache/datafusion-python.git
-<span class="c1"># create the virtual enviornment</span>
+<span class="c1"># create the virtual environment</span>
 uv<span class="w"> </span>sync<span class="w"> </span>--dev<span class="w"> </span>--no-install-package<span class="w"> </span>datafusion
 <span class="c1"># activate the environment</span>
 <span class="nb">source</span><span class="w"> </span>.venv/bin/activate
diff --git a/genindex.html b/genindex.html
index 145252ef..ed00262d 100644
--- a/genindex.html
+++ b/genindex.html
@@ -708,6 +708,8 @@
       <li><a href="autoapi/datafusion/dataframe_formatter/index.html#datafusion.dataframe_formatter.DataFrameHtmlFormatter._build_table_container_start">_build_table_container_start() (datafusion.dataframe_formatter.DataFrameHtmlFormatter method)</a>
 </li>
       <li><a href="autoapi/datafusion/dataframe_formatter/index.html#datafusion.dataframe_formatter.DataFrameHtmlFormatter._build_table_header">_build_table_header() (datafusion.dataframe_formatter.DataFrameHtmlFormatter method)</a>
+</li>
+      <li><a href="autoapi/datafusion/context/index.html#datafusion.context.SessionContext._convert_file_sort_order">_convert_file_sort_order() (datafusion.context.SessionContext static method)</a>
 </li>
       <li><a href="autoapi/datafusion/context/index.html#datafusion.context.SessionContext._convert_table_partition_cols">_convert_table_partition_cols() (datafusion.context.SessionContext static method)</a>
 </li>
@@ -883,10 +885,10 @@
         <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.alias">(in module datafusion.functions)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.allow_single_file_parallelism">allow_single_file_parallelism (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id7">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.allow_single_file_parallelism">allow_single_file_parallelism (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.allow_single_file_parallelism">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id8">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.allow_single_file_parallelism">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/object_store/index.html#datafusion.object_store.AmazonS3">AmazonS3 (in module datafusion.object_store)</a>
@@ -1113,38 +1115,38 @@
 </li>
       <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.bit_xor">bit_xor() (in module datafusion.functions)</a>
 </li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.bloom_filter_enabled">bloom_filter_enabled (datafusion.dataframe.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id0">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.bloom_filter_enabled">bloom_filter_enabled (datafusion.dataframe.ParquetColumnOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.bloom_filter_enabled">(datafusion.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id0">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.bloom_filter_enabled">(datafusion.ParquetColumnOptions attribute)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.bloom_filter_fpp">bloom_filter_fpp (datafusion.dataframe.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id1">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.bloom_filter_fpp">bloom_filter_fpp (datafusion.dataframe.ParquetColumnOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.bloom_filter_fpp">(datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id8">[1]</a>
+        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.bloom_filter_fpp">(datafusion.dataframe.ParquetWriterOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.bloom_filter_fpp">(datafusion.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id2">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.bloom_filter_fpp">(datafusion.ParquetColumnOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.bloom_filter_fpp">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id9">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.bloom_filter_fpp">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
   </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.bloom_filter_ndv">bloom_filter_ndv (datafusion.dataframe.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id2">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.bloom_filter_ndv">bloom_filter_ndv (datafusion.dataframe.ParquetColumnOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.bloom_filter_ndv">(datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id9">[1]</a>
+        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.bloom_filter_ndv">(datafusion.dataframe.ParquetWriterOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.bloom_filter_ndv">(datafusion.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id3">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.bloom_filter_ndv">(datafusion.ParquetColumnOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.bloom_filter_ndv">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id10">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.bloom_filter_ndv">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.bloom_filter_on_write">bloom_filter_on_write (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id10">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.bloom_filter_on_write">bloom_filter_on_write (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.bloom_filter_on_write">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id11">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.bloom_filter_on_write">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.bool_and">bool_and() (in module datafusion.functions)</a>
@@ -1297,10 +1299,10 @@
         <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.column">(datafusion.expr.Expr static method)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.column_index_truncate_length">column_index_truncate_length (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id11">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.column_index_truncate_length">column_index_truncate_length (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.column_index_truncate_length">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id12">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.column_index_truncate_length">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
   </ul></td>
@@ -1311,22 +1313,18 @@
         <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Expr.column_name">(datafusion.expr.Expr method)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.column_specific_options">column_specific_options (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id12">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.column_specific_options">column_specific_options (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.column_specific_options">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id13">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.column_specific_options">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.Compression">Compression (class in datafusion.dataframe)</a>
 </li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.compression">compression (datafusion.dataframe.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id3">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.compression">compression (datafusion.dataframe.ParquetColumnOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.compression">(datafusion.dataframe.ParquetWriterOptions attribute)</a>
-</li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.compression">(datafusion.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id4">[1]</a>
-</li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.compression">(datafusion.ParquetWriterOptions attribute)</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.compression">(datafusion.ParquetColumnOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.concat">concat() (in module datafusion.functions)</a>
@@ -1401,10 +1399,10 @@
 </li>
       <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.CreateCatalogSchema">CreateCatalogSchema (in module datafusion.expr)</a>
 </li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.created_by">created_by (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id13">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.created_by">created_by (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.created_by">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id14">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.created_by">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.CreateExternalTable">CreateExternalTable (in module datafusion.expr)</a>
@@ -1435,16 +1433,16 @@
 <h2 id="D">D</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.data_page_row_count_limit">data_page_row_count_limit (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id14">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.data_page_row_count_limit">data_page_row_count_limit (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.data_page_row_count_limit">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id15">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.data_page_row_count_limit">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.data_pagesize_limit">data_pagesize_limit (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id15">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.data_pagesize_limit">data_pagesize_limit (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.data_pagesize_limit">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id16">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.data_pagesize_limit">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/index.html#datafusion.Database">Database (class in datafusion)</a>
@@ -1649,20 +1647,20 @@
 </li>
       <li><a href="autoapi/datafusion/unparser/index.html#datafusion.unparser.Dialect.dialect">dialect (datafusion.unparser.Dialect attribute)</a>
 </li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.dictionary_enabled">dictionary_enabled (datafusion.dataframe.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id4">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.dictionary_enabled">dictionary_enabled (datafusion.dataframe.ParquetColumnOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.dictionary_enabled">(datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id16">[1]</a>
+        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.dictionary_enabled">(datafusion.dataframe.ParquetWriterOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.dictionary_enabled">(datafusion.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id5">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.dictionary_enabled">(datafusion.ParquetColumnOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.dictionary_enabled">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id17">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.dictionary_enabled">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.dictionary_page_size_limit">dictionary_page_size_limit (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id17">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.dictionary_page_size_limit">dictionary_page_size_limit (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.dictionary_page_size_limit">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id18">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.dictionary_page_size_limit">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.digest">digest() (in module datafusion.functions)</a>
@@ -1757,19 +1755,23 @@
         <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.encode">(in module datafusion.functions)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.encoding">encoding (datafusion.dataframe.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id5">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.encoding">encoding (datafusion.dataframe.ParquetColumnOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.encoding">(datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id18">[1]</a>
+        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.encoding">(datafusion.dataframe.ParquetWriterOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.encoding">(datafusion.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id6">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.encoding">(datafusion.ParquetColumnOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.encoding">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id19">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.encoding">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.CaseBuilder.end">end() (datafusion.expr.CaseBuilder method)</a>
 </li>
       <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.ends_with">ends_with() (in module datafusion.functions)</a>
+</li>
+      <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.ensure_expr">ensure_expr() (in module datafusion.expr)</a>
+</li>
+      <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.ensure_expr_list">ensure_expr_list() (in module datafusion.expr)</a>
 </li>
       <li><a href="autoapi/datafusion/index.html#datafusion.Accumulator.evaluate">evaluate() (datafusion.Accumulator method)</a>
 
@@ -1830,6 +1832,8 @@
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.SortExpr.expr">expr() (datafusion.expr.SortExpr method)</a>
+</li>
+      <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.EXPR_TYPE_ERROR">EXPR_TYPE_ERROR (in module datafusion.expr)</a>
 </li>
       <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Extension">Extension (in module datafusion.expr)</a>
 </li>
@@ -2381,24 +2385,24 @@
 </li>
       <li><a href="autoapi/datafusion/dataframe_formatter/index.html#datafusion.dataframe_formatter.DataFrameHtmlFormatter.max_memory_bytes">max_memory_bytes (datafusion.dataframe_formatter.DataFrameHtmlFormatter attribute)</a>
 </li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.max_row_group_size">max_row_group_size (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id19">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.max_row_group_size">max_row_group_size (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.max_row_group_size">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id20">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.max_row_group_size">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/dataframe_formatter/index.html#datafusion.dataframe_formatter.DataFrameHtmlFormatter.max_width">max_width (datafusion.dataframe_formatter.DataFrameHtmlFormatter attribute)</a>
 </li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">maximum_buffered_record_batches_per_stream (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id20">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">maximum_buffered_record_batches_per_stream (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id21">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.maximum_buffered_record_batches_per_stream">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.maximum_parallel_row_group_writers">maximum_parallel_row_group_writers (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id21">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.maximum_parallel_row_group_writers">maximum_parallel_row_group_writers (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.maximum_parallel_row_group_writers">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id22">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.maximum_parallel_row_group_writers">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/index.html#datafusion.Expr.md5">md5() (datafusion.Expr method)</a>
@@ -3016,10 +3020,10 @@
         <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.sinh">(in module datafusion.functions)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.skip_arrow_metadata">skip_arrow_metadata (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id22">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.skip_arrow_metadata">skip_arrow_metadata (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.skip_arrow_metadata">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id23">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.skip_arrow_metadata">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.Compression.SNAPPY">SNAPPY (datafusion.dataframe.Compression attribute)</a>
@@ -3035,6 +3039,8 @@
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.SortExpr">SortExpr (class in datafusion.expr)</a>
+</li>
+      <li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.SortKey">SortKey (in module datafusion.expr)</a>
 </li>
       <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.split_part">split_part() (in module datafusion.functions)</a>
 </li>
@@ -3068,20 +3074,20 @@
         <li><a href="autoapi/datafusion/user_defined/index.html#datafusion.user_defined.Accumulator.state">(datafusion.user_defined.Accumulator method)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.statistics_enabled">statistics_enabled (datafusion.dataframe.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id6">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetColumnOptions.statistics_enabled">statistics_enabled (datafusion.dataframe.ParquetColumnOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.statistics_enabled">(datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id23">[1]</a>
+        <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.statistics_enabled">(datafusion.dataframe.ParquetWriterOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.statistics_enabled">(datafusion.ParquetColumnOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id7">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetColumnOptions.statistics_enabled">(datafusion.ParquetColumnOptions attribute)</a>
 </li>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.statistics_enabled">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id24">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.statistics_enabled">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.statistics_truncate_length">statistics_truncate_length (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id24">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.statistics_truncate_length">statistics_truncate_length (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.statistics_truncate_length">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id25">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.statistics_truncate_length">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.stddev">stddev() (in module datafusion.functions)</a>
@@ -3590,10 +3596,10 @@
         <li><a href="autoapi/datafusion/index.html#datafusion.RuntimeEnvBuilder.with_unbounded_memory_pool">(datafusion.RuntimeEnvBuilder method)</a>
 </li>
       </ul></li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.write_batch_size">write_batch_size (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id25">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.write_batch_size">write_batch_size (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.write_batch_size">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id26">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.write_batch_size">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
       <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame.write_csv">write_csv() (datafusion.dataframe.DataFrame method)</a>
@@ -3604,10 +3610,10 @@
 </li>
       <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame.write_parquet_with_options">write_parquet_with_options() (datafusion.dataframe.DataFrame method)</a>
 </li>
-      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.writer_version">writer_version (datafusion.dataframe.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/dataframe/index.html#id26">[1]</a>
+      <li><a href="autoapi/datafusion/dataframe/index.html#datafusion.dataframe.ParquetWriterOptions.writer_version">writer_version (datafusion.dataframe.ParquetWriterOptions attribute)</a>
 
       <ul>
-        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.writer_version">(datafusion.ParquetWriterOptions attribute)</a>, <a href="autoapi/datafusion/index.html#id27">[1]</a>
+        <li><a href="autoapi/datafusion/index.html#datafusion.ParquetWriterOptions.writer_version">(datafusion.ParquetWriterOptions attribute)</a>
 </li>
       </ul></li>
   </ul></td>
diff --git a/objects.inv b/objects.inv
index e1fa1a73..827143a5 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/searchindex.js b/searchindex.js
index 081285ea..f833720a 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"API Reference": [[18, null]], "Additional Resources": [[36, "additional-resources"]], "Aggregate Functions": [[23, "aggregate-functions"], [30, "aggregate-functions"], [32, "aggregate-functions"]], "Aggregation": [[23, null]], "Alternative Approach": [[19, "alternative-approach"]], "Arrays": [[25, "arrays"]], "Arrow": [[38, null]], "Attributes": [[4, "attributes"], [7, "attributes"], [12, "attributes"], [17, "attributes"]], "Available Functions": [[32, "av [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"API Reference": [[18, null]], "Additional Resources": [[36, "additional-resources"]], "Aggregate Functions": [[23, "aggregate-functions"], [30, "aggregate-functions"], [32, "aggregate-functions"]], "Aggregation": [[23, null]], "Alternative Approach": [[19, "alternative-approach"]], "Apache Iceberg": [[34, "apache-iceberg"]], "Arrays": [[25, "arrays"]], "Arrow": [[38, null]], "Attributes": [[4, "attributes"], [7, "attributes"], [12, "attributes"], [17, "att [...]
\ No newline at end of file
diff --git a/user-guide/common-operations/aggregations.html b/user-guide/common-operations/aggregations.html
index f9fe3ca3..e4326f10 100644
--- a/user-guide/common-operations/aggregations.html
+++ b/user-guide/common-operations/aggregations.html
@@ -610,15 +610,15 @@ Pokemon set. Since there will be many entries of <code class="docutils literal n
 <span class="o">+----------+--------------------------------------------------+</span>
 <span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span>   <span class="o">|</span> <span class="n">Type</span> <span class="mi">2</span> <span class="n">List</span>                                      <span class="o">|</span>
 <span class="o">+----------+--------------------------------------------------+</span>
-<span class="o">|</span> <span class="n">Bug</span>      <span class="o">|</span> <span class="p">[,</span> <span class="n">Grass</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                        <span class="o">|</span>
-<span class="o">|</span> <span class="n">Poison</span>   <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">,</span> <span class="p">]</span>                               <span class="o">|</span>
+<span class="o">|</span> <span class="n">Bug</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">,</span> <span class="p">,</span> <span class="n">Grass</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                        <span class="o">|</span>
+<span class="o">|</span> <span class="n">Poison</span>   <span class="o">|</span> <span class="p">[</span><span class="n">Ground</span><span class="p">,</span> <span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                               <span class="o">|</span>
 <span class="o">|</span> <span class="n">Electric</span> <span class="o">|</span> <span class="p">[</span><span class="n">Steel</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="p">]</span>                                <span class="o">|</span>
 <span class="o">|</span> <span class="n">Fairy</span>    <span class="o">|</span> <span class="p">[]</span>                                               <span class="o">|</span>
-<span class="o">|</span> <span class="n">Normal</span>   <span class="o">|</span> <span class="p">[</span><span class="n">Fairy</span><span class="p">,</span> <span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                                <span class="o">|</span>
-<span class="o">|</span> <span class="n">Ice</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span>                                <span class="o">|</span>
+<span class="o">|</span> <span class="n">Normal</span>   <span class="o">|</span> <span class="p">[,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Fairy</span><span class="p">]</span>                                <span class="o">|</span>
+<span class="o">|</span> <span class="n">Ice</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Psychic</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                                <span class="o">|</span>
 <span class="o">|</span> <span class="n">Grass</span>    <span class="o">|</span> <span class="p">[,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Poison</span><span class="p">]</span>                              <span class="o">|</span>
-<span class="o">|</span> <span class="n">Fire</span>     <span class="o">|</span> <span class="p">[,</span> <span class="n">Dragon</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                               <span class="o">|</span>
-<span class="o">|</span> <span class="n">Water</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Fighting</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Dark</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Fire</span>     <span class="o">|</span> <span class="p">[,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Dragon</span><span class="p">]</span>                               <span class="o">|</span>
+<span class="o">|</span> <span class="n">Water</span>    <span class="o">|</span> <span class="p">[,</span> <span class="n">Dark</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">]</span> <span class="o">|</span>
 <span class="o">|</span> <span class="n">Ground</span>   <span class="o">|</span> <span class="p">[,</span> <span class="n">Rock</span><span class="p">]</span>                                         <span class="o">|</span>
 <span class="o">+----------+--------------------------------------------------+</span>
 <span class="n">Data</span> <span class="n">truncated</span><span class="o">.</span>
@@ -634,15 +634,15 @@ entries entirely removed. The second is we can use the <code class="docutils lit
 <span class="o">+----------+------------------------------------------------+</span>
 <span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span>   <span class="o">|</span> <span class="n">Type</span> <span class="mi">2</span> <span class="n">List</span>                                    <span class="o">|</span>
 <span class="o">+----------+------------------------------------------------+</span>
-<span class="o">|</span> <span class="n">Bug</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Grass</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Poison</span><span class="p">]</span>                        <span class="o">|</span>
+<span class="o">|</span> <span class="n">Bug</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">,</span> <span class="n">Grass</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                        <span class="o">|</span>
 <span class="o">|</span> <span class="n">Poison</span>   <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">]</span>                               <span class="o">|</span>
 <span class="o">|</span> <span class="n">Electric</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Steel</span><span class="p">]</span>                                <span class="o">|</span>
 <span class="o">|</span> <span class="n">Normal</span>   <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Fairy</span><span class="p">]</span>                                <span class="o">|</span>
 <span class="o">|</span> <span class="n">Ice</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Psychic</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                              <span class="o">|</span>
-<span class="o">|</span> <span class="n">Grass</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Psychic</span><span class="p">,</span> <span class="n">Poison</span><span class="p">]</span>                              <span class="o">|</span>
+<span class="o">|</span> <span class="n">Grass</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span>                              <span class="o">|</span>
 <span class="o">|</span> <span class="n">Fire</span>     <span class="o">|</span> <span class="p">[</span><span class="n">Dragon</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                               <span class="o">|</span>
-<span class="o">|</span> <span class="n">Water</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Fighting</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Dark</span><span class="p">,</span> <span class="n">Ice</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Rock</span>     <span class="o">|</span> <span class="p">[</span><span class="n">Water</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">]</span>                        <span class="o">|</span>
+<span class="o">|</span> <span class="n">Water</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Dark</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Rock</span>     <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">,</span> <span class="n">Water</span><span class="p">]</span>                        <span class="o">|</span>
 <span class="o">|</span> <span class="n">Ghost</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">]</span>                                       <span class="o">|</span>
 <span class="o">+----------+------------------------------------------------+</span>
 <span class="n">Data</span> <span class="n">truncated</span><span class="o">.</span>
@@ -653,15 +653,15 @@ entries entirely removed. The second is we can use the <code class="docutils lit
 <span class="o">+----------+------------------------------------------------+</span>
 <span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span>   <span class="o">|</span> <span class="n">Type</span> <span class="mi">2</span> <span class="n">List</span>                                    <span class="o">|</span>
 <span class="o">+----------+------------------------------------------------+</span>
-<span class="o">|</span> <span class="n">Bug</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">,</span> <span class="n">Grass</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                        <span class="o">|</span>
+<span class="o">|</span> <span class="n">Bug</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Grass</span><span class="p">,</span> <span class="n">Poison</span><span class="p">]</span>                        <span class="o">|</span>
 <span class="o">|</span> <span class="n">Poison</span>   <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">]</span>                               <span class="o">|</span>
-<span class="o">|</span> <span class="n">Electric</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Steel</span><span class="p">]</span>                                <span class="o">|</span>
+<span class="o">|</span> <span class="n">Electric</span> <span class="o">|</span> <span class="p">[</span><span class="n">Steel</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                                <span class="o">|</span>
 <span class="o">|</span> <span class="n">Fairy</span>    <span class="o">|</span>                                                <span class="o">|</span>
 <span class="o">|</span> <span class="n">Normal</span>   <span class="o">|</span> <span class="p">[</span><span class="n">Fairy</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                                <span class="o">|</span>
 <span class="o">|</span> <span class="n">Ice</span>      <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span>                              <span class="o">|</span>
 <span class="o">|</span> <span class="n">Grass</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Psychic</span><span class="p">,</span> <span class="n">Poison</span><span class="p">]</span>                              <span class="o">|</span>
 <span class="o">|</span> <span class="n">Fire</span>     <span class="o">|</span> <span class="p">[</span><span class="n">Dragon</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span>                               <span class="o">|</span>
-<span class="o">|</span> <span class="n">Water</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Dark</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Water</span>    <span class="o">|</span> <span class="p">[</span><span class="n">Ice</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">,</span> <span class="n">Dark</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
 <span class="o">|</span> <span class="n">Ground</span>   <span class="o">|</span> <span class="p">[</span><span class="n">Rock</span><span class="p">]</span>                                         <span class="o">|</span>
 <span class="o">+----------+------------------------------------------------+</span>
 <span class="n">Data</span> <span class="n">truncated</span><span class="o">.</span>
diff --git a/user-guide/common-operations/basic-info.html b/user-guide/common-operations/basic-info.html
index c4d5beae..5f5186e7 100644
--- a/user-guide/common-operations/basic-info.html
+++ b/user-guide/common-operations/basic-info.html
@@ -446,11 +446,11 @@
 <span class="o">+-----+---------+--------+--------+</span>
 <span class="o">|</span> <span class="n">nrs</span> <span class="o">|</span> <span class="n">names</span>   <span class="o">|</span> <span class="n">random</span> <span class="o">|</span> <span class="n">groups</span> <span class="o">|</span>
 <span class="o">+-----+---------+--------+--------+</span>
-<span class="o">|</span> <span class="mi">1</span>   <span class="o">|</span> <span class="n">python</span>  <span class="o">|</span> <span class="mi">653</span>    <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2</span>   <span class="o">|</span> <span class="n">ruby</span>    <span class="o">|</span> <span class="mi">417</span>    <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="mi">3</span>   <span class="o">|</span> <span class="n">java</span>    <span class="o">|</span> <span class="mi">408</span>    <span class="o">|</span> <span class="n">B</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="mi">4</span>   <span class="o">|</span> <span class="n">haskell</span> <span class="o">|</span> <span class="mi">196</span>    <span class="o">|</span> <span class="n">C</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="mi">5</span>   <span class="o">|</span> <span class="n">go</span>      <span class="o">|</span> <span class="mi">840</span>    <span class="o">|</span> <span class="n">B</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="mi">1</span>   <span class="o">|</span> <span class="n">python</span>  <span class="o">|</span> <span class="mi">968</span>    <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2</span>   <span class="o">|</span> <span class="n">ruby</span>    <span class="o">|</span> <span class="mi">979</span>    <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="mi">3</span>   <span class="o">|</span> <span class="n">java</span>    <span class="o">|</span> <span class="mi">177</span>    <span class="o">|</span> <span class="n">B</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="mi">4</span>   <span class="o">|</span> <span class="n">haskell</span> <span class="o">|</span> <span class="mi">423</span>    <span class="o">|</span> <span class="n">C</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="mi">5</span>   <span class="o">|</span> <span class="n">go</span>      <span class="o">|</span> <span class="mi">795</span>    <span class="o">|</span> <span class="n">B</span>      <span class="o">|</span>
 <span class="o">+-----+---------+--------+--------+</span>
 </pre></div>
 </div>
@@ -461,8 +461,8 @@
 <span class="o">+-----+--------+--------+--------+</span>
 <span class="o">|</span> <span class="n">nrs</span> <span class="o">|</span> <span class="n">names</span>  <span class="o">|</span> <span class="n">random</span> <span class="o">|</span> <span class="n">groups</span> <span class="o">|</span>
 <span class="o">+-----+--------+--------+--------+</span>
-<span class="o">|</span> <span class="mi">1</span>   <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">653</span>    <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2</span>   <span class="o">|</span> <span class="n">ruby</span>   <span class="o">|</span> <span class="mi">417</span>    <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="mi">1</span>   <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">968</span>    <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2</span>   <span class="o">|</span> <span class="n">ruby</span>   <span class="o">|</span> <span class="mi">979</span>    <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
 <span class="o">+-----+--------+--------+--------+</span>
 </pre></div>
 </div>
@@ -480,28 +480,28 @@ passing them to an Arrow table, and then converting them to a pandas DataFrame.<
 <div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">to_pandas</span><span class="p">()</span>
 <span class="n">Out</span><span class="p">[</span><span class="mi">8</span><span class="p">]:</span> 
    <span class="n">nrs</span>    <span class="n">names</span>  <span class="n">random</span> <span class="n">groups</span>
-<span class="mi">0</span>    <span class="mi">1</span>   <span class="n">python</span>     <span class="mi">653</span>      <span class="n">A</span>
-<span class="mi">1</span>    <span class="mi">2</span>     <span class="n">ruby</span>     <span class="mi">417</span>      <span class="n">A</span>
-<span class="mi">2</span>    <span class="mi">3</span>     <span class="n">java</span>     <span class="mi">408</span>      <span class="n">B</span>
-<span class="mi">3</span>    <span class="mi">4</span>  <span class="n">haskell</span>     <span class="mi">196</span>      <span class="n">C</span>
-<span class="mi">4</span>    <span class="mi">5</span>       <span class="n">go</span>     <span class="mi">840</span>      <span class="n">B</span>
+<span class="mi">0</span>    <span class="mi">1</span>   <span class="n">python</span>     <span class="mi">968</span>      <span class="n">A</span>
+<span class="mi">1</span>    <span class="mi">2</span>     <span class="n">ruby</span>     <span class="mi">979</span>      <span class="n">A</span>
+<span class="mi">2</span>    <span class="mi">3</span>     <span class="n">java</span>     <span class="mi">177</span>      <span class="n">B</span>
+<span class="mi">3</span>    <span class="mi">4</span>  <span class="n">haskell</span>     <span class="mi">423</span>      <span class="n">C</span>
+<span class="mi">4</span>    <span class="mi">5</span>       <span class="n">go</span>     <span class="mi">795</span>      <span class="n">B</span>
 </pre></div>
 </div>
 <p><a class="reference internal" href="../../autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame.describe" title="datafusion.dataframe.DataFrame.describe"><code class="xref py py-func docutils literal notranslate"><span class="pre">describe()</span></code></a> shows a quick statistic summary of your data:</p>
 <div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">describe</span><span class="p">()</span>
 <span class="n">Out</span><span class="p">[</span><span class="mi">9</span><span class="p">]:</span> 
 <span class="n">DataFrame</span><span class="p">()</span>
-<span class="o">+------------+--------------------+-------+--------------------+--------+</span>
-<span class="o">|</span> <span class="n">describe</span>   <span class="o">|</span> <span class="n">nrs</span>                <span class="o">|</span> <span class="n">names</span> <span class="o">|</span> <span class="n">random</span>             <span class="o">|</span> <span class="n">groups</span> <span class="o">|</span>
-<span class="o">+------------+--------------------+-------+--------------------+--------+</span>
-<span class="o">|</span> <span class="n">count</span>      <span class="o">|</span> <span class="mf">5.0</span>                <span class="o">|</span> <span class="mi">5</span>     <span class="o">|</span> <span class="mf">5.0</span>                <span class="o">|</span> <span class="mi">5</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="n">null_count</span> <span class="o">|</span> <span class="mf">0.0</span>                <span class="o">|</span> <span class="mi">0</span>     <span class="o">|</span> <span class="mf">0.0</span>                <span class="o">|</span> <span class="mi">0</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="n">mean</span>       <span class="o">|</span> <span class="mf">3.0</span>                <span class="o">|</span> <span class="n">null</span>  <span class="o">|</span> <span class="mf">502.8</span>              <span class="o">|</span> <span class="n">null</span>   <span class="o">|</span>
-<span class="o">|</span> <span class="n">std</span>        <span class="o">|</span> <span class="mf">1.5811388300841898</span> <span class="o">|</span> <span class="n">null</span>  <span class="o">|</span> <span class="mf">248.36404731764216</span> <span class="o">|</span> <span class="n">null</span>   <span class="o">|</span>
-<span class="o">|</span> <span class="nb">min</span>        <span class="o">|</span> <span class="mf">1.0</span>                <span class="o">|</span> <span class="n">go</span>    <span class="o">|</span> <span class="mf">196.0</span>              <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="nb">max</span>        <span class="o">|</span> <span class="mf">5.0</span>                <span class="o">|</span> <span class="n">ruby</span>  <span class="o">|</span> <span class="mf">840.0</span>              <span class="o">|</span> <span class="n">C</span>      <span class="o">|</span>
-<span class="o">|</span> <span class="n">median</span>     <span class="o">|</span> <span class="mf">3.0</span>                <span class="o">|</span> <span class="n">null</span>  <span class="o">|</span> <span class="mf">417.0</span>              <span class="o">|</span> <span class="n">null</span>   <span class="o">|</span>
-<span class="o">+------------+--------------------+-------+--------------------+--------+</span>
+<span class="o">+------------+--------------------+-------+-------------------+--------+</span>
+<span class="o">|</span> <span class="n">describe</span>   <span class="o">|</span> <span class="n">nrs</span>                <span class="o">|</span> <span class="n">names</span> <span class="o">|</span> <span class="n">random</span>            <span class="o">|</span> <span class="n">groups</span> <span class="o">|</span>
+<span class="o">+------------+--------------------+-------+-------------------+--------+</span>
+<span class="o">|</span> <span class="n">count</span>      <span class="o">|</span> <span class="mf">5.0</span>                <span class="o">|</span> <span class="mi">5</span>     <span class="o">|</span> <span class="mf">5.0</span>               <span class="o">|</span> <span class="mi">5</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="n">null_count</span> <span class="o">|</span> <span class="mf">0.0</span>                <span class="o">|</span> <span class="mi">0</span>     <span class="o">|</span> <span class="mf">0.0</span>               <span class="o">|</span> <span class="mi">0</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="n">mean</span>       <span class="o">|</span> <span class="mf">3.0</span>                <span class="o">|</span> <span class="n">null</span>  <span class="o">|</span> <span class="mf">668.4</span>             <span class="o">|</span> <span class="n">null</span>   <span class="o">|</span>
+<span class="o">|</span> <span class="n">std</span>        <span class="o">|</span> <span class="mf">1.5811388300841898</span> <span class="o">|</span> <span class="n">null</span>  <span class="o">|</span> <span class="mf">354.9490104226239</span> <span class="o">|</span> <span class="n">null</span>   <span class="o">|</span>
+<span class="o">|</span> <span class="nb">min</span>        <span class="o">|</span> <span class="mf">1.0</span>                <span class="o">|</span> <span class="n">go</span>    <span class="o">|</span> <span class="mf">177.0</span>             <span class="o">|</span> <span class="n">A</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="nb">max</span>        <span class="o">|</span> <span class="mf">5.0</span>                <span class="o">|</span> <span class="n">ruby</span>  <span class="o">|</span> <span class="mf">979.0</span>             <span class="o">|</span> <span class="n">C</span>      <span class="o">|</span>
+<span class="o">|</span> <span class="n">median</span>     <span class="o">|</span> <span class="mf">3.0</span>                <span class="o">|</span> <span class="n">null</span>  <span class="o">|</span> <span class="mf">795.0</span>             <span class="o">|</span> <span class="n">null</span>   <span class="o">|</span>
+<span class="o">+------------+--------------------+-------+-------------------+--------+</span>
 </pre></div>
 </div>
 </section>
diff --git a/user-guide/common-operations/expressions.html b/user-guide/common-operations/expressions.html
index d9c4d351..f2358ca3 100644
--- a/user-guide/common-operations/expressions.html
+++ b/user-guide/common-operations/expressions.html
@@ -497,7 +497,7 @@ examples for the and, or, and not operations.</p>
 <section id="arrays">
 <h2>Arrays<a class="headerlink" href="#arrays" title="Link to this heading">¶</a></h2>
 <p>For columns that contain arrays of values, you can access individual elements of the array by index
-using bracket indexing. This is similar to callling the function
+using bracket indexing. This is similar to calling the function
 <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.array_element" title="datafusion.functions.array_element"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.array_element()</span></code></a>, except that array indexing using brackets is 0 based,
 similar to Python arrays and <code class="docutils literal notranslate"><span class="pre">array_element</span></code> is 1 based indexing to be compatible with other SQL
 approaches.</p>
@@ -523,18 +523,31 @@ approaches.</p>
 <p>Indexing an element of an array via <code class="docutils literal notranslate"><span class="pre">[]</span></code> starts at index 0 whereas
 <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.array_element" title="datafusion.functions.array_element"><code class="xref py py-func docutils literal notranslate"><span class="pre">array_element()</span></code></a> starts at index 1.</p>
 </div>
+<p>Starting in DataFusion 49.0.0 you can also create slices of array elements using
+slice syntax from Python.</p>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">)[</span><span class="mi">1</span><span class="p">:</span><span class="mi">3</span><span class="p">]</ [...]
+<span class="n">Out</span><span class="p">[</span><span class="mi">9</span><span class="p">]:</span> 
+<span class="n">DataFrame</span><span class="p">()</span>
+<span class="o">+---------------------+</span>
+<span class="o">|</span> <span class="n">second_two_elements</span> <span class="o">|</span>
+<span class="o">+---------------------+</span>
+<span class="o">|</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>              <span class="o">|</span>
+<span class="o">|</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">]</span>              <span class="o">|</span>
+<span class="o">+---------------------+</span>
+</pre></div>
+</div>
 <p>To check if an array is empty, you can use the function <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.array_empty" title="datafusion.functions.array_empty"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.array_empty()</span></code></a> or <cite>datafusion.functions.empty</cite>.
 This function returns a boolean indicating whether the array is empty.</p>
-<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span><span class="p">,</span> <span class="n">col</span>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span><span class="p">,</span> <span class="n">col</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="kn">import</span> <span class="n">array_empty</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="kn">import</span> <span class="n">array_empty</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">({</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[[],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi" [...]
+<span class="n">In</span> <span class="p">[</span><span class="mi">13</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">({</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[[],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi" [...]
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">13</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">array_empty</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;is_empty&quot;</spa [...]
-<span class="n">Out</span><span class="p">[</span><span class="mi">13</span><span class="p">]:</span> 
+<span class="n">In</span> <span class="p">[</span><span class="mi">14</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">array_empty</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;is_empty&quot;</spa [...]
+<span class="n">Out</span><span class="p">[</span><span class="mi">14</span><span class="p">]:</span> 
 <span class="n">DataFrame</span><span class="p">()</span>
 <span class="o">+----------+</span>
 <span class="o">|</span> <span class="n">is_empty</span> <span class="o">|</span>
@@ -547,16 +560,16 @@ This function returns a boolean indicating whether the array is empty.</p>
 <p>In this example, the <cite>is_empty</cite> column will contain <cite>True</cite> for the first row and <cite>False</cite> for the second row.</p>
 <p>To get the total number of elements in an array, you can use the function <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.cardinality" title="datafusion.functions.cardinality"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.cardinality()</span></code></a>.
 This function returns an integer indicating the total number of elements in the array.</p>
-<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">14</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span><span class="p">,</span> <span class="n">col</span>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">15</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span><span class="p">,</span> <span class="n">col</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">15</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="kn">import</span> <span class="n">cardinality</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">16</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="kn">import</span> <span class="n">cardinality</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">16</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">17</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">17</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">({</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">], [...]
+<span class="n">In</span> <span class="p">[</span><span class="mi">18</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">({</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">], [...]
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">18</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">cardinality</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;num_elements&quot;< [...]
-<span class="n">Out</span><span class="p">[</span><span class="mi">18</span><span class="p">]:</span> 
+<span class="n">In</span> <span class="p">[</span><span class="mi">19</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">cardinality</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">))</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;num_elements&quot;< [...]
+<span class="n">Out</span><span class="p">[</span><span class="mi">19</span><span class="p">]:</span> 
 <span class="n">DataFrame</span><span class="p">()</span>
 <span class="o">+--------------+</span>
 <span class="o">|</span> <span class="n">num_elements</span> <span class="o">|</span>
@@ -569,16 +582,16 @@ This function returns an integer indicating the total number of elements in the
 <p>In this example, the <cite>num_elements</cite> column will contain <cite>3</cite> for both rows.</p>
 <p>To concatenate two arrays, you can use the function <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.array_cat" title="datafusion.functions.array_cat"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.array_cat()</span></code></a> or <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.array_concat" title="datafusion.functions.array_co [...]
 These functions return a new array that is the concatenation of the input arrays.</p>
-<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">19</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span><span class="p">,</span> <span class="n">col</span>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">20</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span><span class="p">,</span> <span class="n">col</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">20</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="kn">import</span> <span class="n">array_cat</span><span class="p">,</span> <span class="n">array_concat</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">21</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="kn">import</span> <span class="n">array_cat</span><span class="p">,</span> <span class="n">array_concat</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">21</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">22</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">22</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">({</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]] [...]
+<span class="n">In</span> <span class="p">[</span><span class="mi">23</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">({</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]] [...]
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">23</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">array_cat</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;b&quot;</span><span class="p">))</span><span c [...]
-<span class="n">Out</span><span class="p">[</span><span class="mi">23</span><span class="p">]:</span> 
+<span class="n">In</span> <span class="p">[</span><span class="mi">24</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">array_cat</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;b&quot;</span><span class="p">))</span><span c [...]
+<span class="n">Out</span><span class="p">[</span><span class="mi">24</span><span class="p">]:</span> 
 <span class="n">DataFrame</span><span class="p">()</span>
 <span class="o">+--------------------+</span>
 <span class="o">|</span> <span class="n">concatenated_array</span> <span class="o">|</span>
@@ -590,16 +603,16 @@ These functions return a new array that is the concatenation of the input arrays
 <p>In this example, the <cite>concatenated_array</cite> column will contain <cite>[1, 2, 3, 4, 5, 6]</cite>.</p>
 <p>To repeat the elements of an array a specified number of times, you can use the function <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#datafusion.functions.array_repeat" title="datafusion.functions.array_repeat"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.functions.array_repeat()</span></code></a>.
 This function returns a new array with the elements repeated.</p>
-<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">24</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span><span class="p">,</span> <span class="n">col</span><span class="p">,</span> <span class="n">literal</span>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">25</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span><span class="p">,</span> <span class="n">col</span><span class="p">,</span> <span class="n">literal</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">25</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="kn">import</span> <span class="n">array_repeat</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">26</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.functions</span><span class="w"> </span><span class="kn">import</span> <span class="n">array_repeat</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">26</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">27</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">27</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">({</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]] [...]
+<span class="n">In</span> <span class="p">[</span><span class="mi">28</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">({</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]] [...]
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">28</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">array_repeat</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="n">literal</span><span class="p">(</span><span class="mi">2</span><span class="p">))</span><span class= [...]
-<span class="n">Out</span><span class="p">[</span><span class="mi">28</span><span class="p">]:</span> 
+<span class="n">In</span> <span class="p">[</span><span class="mi">29</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">array_repeat</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="n">literal</span><span class="p">(</span><span class="mi">2</span><span class="p">))</span><span class= [...]
+<span class="n">Out</span><span class="p">[</span><span class="mi">29</span><span class="p">]:</span> 
 <span class="n">DataFrame</span><span class="p">()</span>
 <span class="o">+------------------------+</span>
 <span class="o">|</span> <span class="n">repeated_array</span>         <span class="o">|</span>
@@ -614,14 +627,14 @@ This function returns a new array with the elements repeated.</p>
 <h2>Structs<a class="headerlink" href="#structs" title="Link to this heading">¶</a></h2>
 <p>Columns that contain struct elements can be accessed using the bracket notation as if they were
 Python dictionary style objects. This expects a string key as the parameter passed.</p>
-<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">29</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">30</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">30</span><span class="p">]:</span> <span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[{</span><span class="s2">&quot;size&quot;</span><span class="p">:</span> <span class="mi">15</span><span class="p">,</span> <span class="s2">&quot;color&quot;</span><span class="p">:</span> <span class="s2">&quot;green&quot;</span><sp [...]
+<span class="n">In</span> <span class="p">[</span><span class="mi">31</span><span class="p">]:</span> <span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&quot;a&quot;</span><span class="p">:</span> <span class="p">[{</span><span class="s2">&quot;size&quot;</span><span class="p">:</span> <span class="mi">15</span><span class="p">,</span> <span class="s2">&quot;color&quot;</span><span class="p">:</span> <span class="s2">&quot;green&quot;</span><sp [...]
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">31</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">32</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">32</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">)[</span><span class="s2">&quot;size&quot;</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;a_size&quot;< [...]
-<span class="n">Out</span><span class="p">[</span><span class="mi">32</span><span class="p">]:</span> 
+<span class="n">In</span> <span class="p">[</span><span class="mi">33</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">)[</span><span class="s2">&quot;size&quot;</span><span class="p">]</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;a_size&quot;< [...]
+<span class="n">Out</span><span class="p">[</span><span class="mi">33</span><span class="p">]:</span> 
 <span class="n">DataFrame</span><span class="p">()</span>
 <span class="o">+--------+</span>
 <span class="o">|</span> <span class="n">a_size</span> <span class="o">|</span>
@@ -638,17 +651,17 @@ Python dictionary style objects. This expects a string key as the parameter pass
 a wide variety of expressions built up from other expressions. For example, <a class="reference internal" href="../../autoapi/datafusion/expr/index.html#datafusion.expr.Expr.alias" title="datafusion.expr.Expr.alias"><code class="xref py py-func docutils literal notranslate"><span class="pre">alias()</span></code></a> is a function that takes
 as it input a single expression and returns an expression in which the name of the expression has changed.</p>
 <p>The following example shows a series of expressions that are built up from functions operating on expressions.</p>
-<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">33</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">34</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">34</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">column</span><span class="p">,</span> <span class="n">lit</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">35</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">column</span><span class="p">,</span> <span class="n">lit</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">35</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">functions</span> <span class="k">as</span> <span class="n">f</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">36</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">functions</span> <span class="k">as</span> <span class="n">f</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">36</span><span class="p">]:</span> <span class="kn">import</span><span class="w"> </span><span class="nn">random</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">37</span><span class="p">]:</span> <span class="kn">import</span><span class="w"> </span><span class="nn">random</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">37</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">38</span><span class="p">]:</span> <span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">38</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">(</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">39</span><span class="p">]:</span> <span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">from_pydict</span><span class="p">(</span>
    <span class="o">....</span><span class="p">:</span>     <span class="p">{</span>
    <span class="o">....</span><span class="p">:</span>         <span class="s2">&quot;name&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;Albert&quot;</span><span class="p">,</span> <span class="s2">&quot;Becca&quot;</span><span class="p">,</span> <span class="s2">&quot;Carlos&quot;</span><span class="p">,</span> <span class="s2">&quot;Dante&quot;</span><span class="p">],</span>
    <span class="o">....</span><span class="p">:</span>         <span class="s2">&quot;age&quot;</span><span class="p">:</span> <span class="p">[</span><span class="mi">42</span><span class="p">,</span> <span class="mi">67</span><span class="p">,</span> <span class="mi">27</span><span class="p">,</span> <span class="mi">71</span><span class="p">],</span>
@@ -658,20 +671,20 @@ as it input a single expression and returns an expression in which the name of t
    <span class="o">....</span><span class="p">:</span> <span class="p">)</span>
    <span class="o">....</span><span class="p">:</span> 
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">39</span><span class="p">]:</span> <span class="n">age_col</span> <span class="o">=</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;age&quot;</span><span class="p">)</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">40</span><span class="p">]:</span> <span class="n">age_col</span> <span class="o">=</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;age&quot;</span><span class="p">)</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">40</span><span class="p">]:</span> <span class="n">renamed_age</span> <span class="o">=</span> <span class="n">age_col</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;age_in_years&quot;</span><span class="p">)</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">41</span><span class="p">]:</span> <span class="n">renamed_age</span> <span class="o">=</span> <span class="n">age_col</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;age_in_years&quot;</span><span class="p">)</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">41</span><span class="p">]:</span> <span class="n">start_age</span> <span class="o">=</span> <span class="n">age_col</span> <span class="o">-</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;years_in_position&quot;</span><span class="p">)</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">42</span><span class="p">]:</span> <span class="n">start_age</span> <span class="o">=</span> <span class="n">age_col</span> <span class="o">-</span> <span class="n">col</span><span class="p">(</span><span class="s2">&quot;years_in_position&quot;</span><span class="p">)</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">42</span><span class="p">]:</span> <span class="n">started_young</span> <span class="o">=</span> <span class="n">start_age</span> <span class="o">&lt;</span> <span class="n">lit</span><span class="p">(</span><span class="mi">18</span><span class="p">)</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">43</span><span class="p">]:</span> <span class="n">started_young</span> <span class="o">=</span> <span class="n">start_age</span> <span class="o">&lt;</span> <span class="n">lit</span><span class="p">(</span><span class="mi">18</span><span class="p">)</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">43</span><span class="p">]:</span> <span class="n">can_retire</span> <span class="o">=</span> <span class="n">age_col</span> <span class="o">&gt;</span> <span class="n">lit</span><span class="p">(</span><span class="mi">65</span><span class="p">)</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">44</span><span class="p">]:</span> <span class="n">can_retire</span> <span class="o">=</span> <span class="n">age_col</span> <span class="o">&gt;</span> <span class="n">lit</span><span class="p">(</span><span class="mi">65</span><span class="p">)</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">44</span><span class="p">]:</span> <span class="n">long_timer</span> <span class="o">=</span> <span class="n">started_young</span> <span class="o">&amp;</span> <span class="n">can_retire</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">45</span><span class="p">]:</span> <span class="n">long_timer</span> <span class="o">=</span> <span class="n">started_young</span> <span class="o">&amp;</span> <span class="n">can_retire</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">45</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">long_timer</span><span class="p">)</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;name&quot;</span><span class="p">),</span> <span class="n">renamed_age</span><spa [...]
-<span class="n">Out</span><span class="p">[</span><span class="mi">45</span><span class="p">]:</span> 
+<span class="n">In</span> <span class="p">[</span><span class="mi">46</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">long_timer</span><span class="p">)</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;name&quot;</span><span class="p">),</span> <span class="n">renamed_age</span><spa [...]
+<span class="n">Out</span><span class="p">[</span><span class="mi">46</span><span class="p">]:</span> 
 <span class="n">DataFrame</span><span class="p">()</span>
 <span class="o">+-------+--------------+-------------------+</span>
 <span class="o">|</span> <span class="n">name</span>  <span class="o">|</span> <span class="n">age_in_years</span> <span class="o">|</span> <span class="n">years_in_position</span> <span class="o">|</span>
diff --git a/user-guide/common-operations/functions.html b/user-guide/common-operations/functions.html
index 916147c1..cfc85464 100644
--- a/user-guide/common-operations/functions.html
+++ b/user-guide/common-operations/functions.html
@@ -556,16 +556,16 @@ In here we will cover some of the more popular use cases. If you want to view al
 <span class="o">+--------------------------------+</span>
 <span class="o">|</span> <span class="n">now</span><span class="p">()</span>                          <span class="o">|</span>
 <span class="o">+--------------------------------+</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="n">T01</span><span class="p">:</span><span class="mi">44</span><span class="p">:</span><span class="mf">48.517857695</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2025</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">17</span><span class="n">T19</span><span class="p">:</span><span class="mi">32</span><span class="p">:</span><span class="mf">40.245872298</span><span class="n">Z</span> <span class="o">|</span>
 <span class="o">+--------------------------------+</span>
 <span class="n">Data</span> <span class="n">truncated</span><span class="o">.</span>
 </pre></div>
diff --git a/user-guide/common-operations/windows.html b/user-guide/common-operations/windows.html
index 4ac5a095..ac2682ab 100644
--- a/user-guide/common-operations/windows.html
+++ b/user-guide/common-operations/windows.html
@@ -470,11 +470,11 @@
 <p>In this section you will learn about window functions. A window function utilizes values from one or
 multiple rows to produce a result for each individual row, unlike an aggregate function that
 provides a single value for multiple rows.</p>
-<p>The window functions are availble in the <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#module-datafusion.functions" title="datafusion.functions"><code class="xref py py-mod docutils literal notranslate"><span class="pre">functions</span></code></a> module.</p>
+<p>The window functions are available in the <a class="reference internal" href="../../autoapi/datafusion/functions/index.html#module-datafusion.functions" title="datafusion.functions"><code class="xref py py-mod docutils literal notranslate"><span class="pre">functions</span></code></a> module.</p>
 <p>We’ll use the pokemon dataset (from Ritchie Vink) in the following examples.</p>
 <div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span>
+<span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span><span class="p">,</span> <span class="n">lit</span>
 
 <span class="n">In</span> <span class="p">[</span><span class="mi">3</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">functions</span> <span class="k">as</span> <span class="n">f</span>
 
@@ -590,8 +590,8 @@ If you do not specify a Window Frame, the frame will be set depending on the fol
 criteria.</p>
 <ul class="simple">
 <li><p>If an <code class="docutils literal notranslate"><span class="pre">order_by</span></code> clause is set, the default window frame is defined as the rows between
-unbounded preceeding and the current row.</p></li>
-<li><p>If an <code class="docutils literal notranslate"><span class="pre">order_by</span></code> is not set, the default frame is defined as the rows betwene unbounded
+unbounded preceding and the current row.</p></li>
+<li><p>If an <code class="docutils literal notranslate"><span class="pre">order_by</span></code> is not set, the default frame is defined as the rows between unbounded
 and unbounded following (the entire partition).</p></li>
 </ul>
 <p>Window Frames are defined by three parameters: unit type, starting bound, and ending bound.</p>
@@ -606,17 +606,15 @@ parameter.</p></li>
 <code class="docutils literal notranslate"><span class="pre">order_by</span></code> clause.</p></li>
 </ul>
 <p>In this example we perform a “rolling average” of the speed of the current Pokemon and the
-two preceeding rows.</p>
-<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.expr</span><span class="w"> </span><span class="kn">import</span> <span class="n">WindowFrame</span>
+two preceding rows.</p>
+<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">9</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.expr</span><span class="w"> </span><span class="kn">import</span> <span class="n">Window</span><span class="p">,</span> <span class="n">WindowFrame</span>
 
 <span class="n">In</span> <span class="p">[</span><span class="mi">10</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">select</span><span class="p">(</span>
    <span class="o">....</span><span class="p">:</span>     <span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Name&quot;&#39;</span><span class="p">),</span>
    <span class="o">....</span><span class="p">:</span>     <span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Speed&quot;&#39;</span><span class="p">),</span>
-   <span class="o">....</span><span class="p">:</span>     <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span><span class="s2">&quot;avg&quot;</span><span class="p">,</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Speed&quot;&#39;</span><span class="p">)],</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="n">order_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Speed&quot;&#39;</span><span class="p">)],</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="n">window_frame</span><span class="o">=</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">&quot;rows&quot;</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
-   <span class="o">....</span><span class="p">:</span>     <span class="p">)</span><span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;Previous Speed&quot;</span><span class="p">)</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="n">f</span><span class="o">.</span><span class="n">avg</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Speed&quot;&#39;</span><span class="p">))</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="o">.</span><span class="n">over</span><span class="p">(</span><span class="n">Window</span><span class="p">(</span><span class="n">window_frame</span><span class="o">=</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">&quot;rows&quot;</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="n">order_ [...]
+   <span class="o">....</span><span class="p">:</span>     <span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;Previous Speed&quot;</span><span class="p">),</span>
    <span class="o">....</span><span class="p">:</span> <span class="p">)</span>
    <span class="o">....</span><span class="p">:</span> 
 <span class="n">Out</span><span class="p">[</span><span class="mi">10</span><span class="p">]:</span> 
@@ -652,21 +650,27 @@ the window frame so that we only process up to the current row.</p>
 it’s <code class="docutils literal notranslate"><span class="pre">Type</span> <span class="pre">2</span></code> column that are null.</p>
 <div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">11</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.common</span><span class="w"> </span><span class="kn">import</span> <span class="n">NullTreatment</span>
 
-<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Type 1&quot;&#39;</span><span class="p">)</span> <span class="o">==</span>  <span class="n">lit</span><span class="p">(</span><span class="s2">&quot;Bug&quot;</span><span class="p">))</span><span class="o">.</s [...]
+<span class="n">In</span> <span class="p">[</span><span class="mi">12</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Type 1&quot;&#39;</span><span class="p">)</span> <span class="o">==</span> <span class="n">lit</span><span class="p">(</span><span class="s2">&quot;Bug&quot;</span><span class="p">))</span><span class="o">.</sp [...]
    <span class="o">....</span><span class="p">:</span>     <span class="s1">&#39;&quot;Name&quot;&#39;</span><span class="p">,</span>
    <span class="o">....</span><span class="p">:</span>     <span class="s1">&#39;&quot;Type 2&quot;&#39;</span><span class="p">,</span>
-   <span class="o">....</span><span class="p">:</span>     <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span><span class="s2">&quot;last_value&quot;</span><span class="p">,</span> <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Type 2&quot;&#39;</span><span class="p">)])</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">window_frame</span><span class="p">(</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">&quot;rows&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Speed&quot;&#39;</span><span class="p">))</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">null_treatment</span><span class="p">(</span><span class="n">NullTreatment</span><span class="o">.</span><span class="n">IGNORE_NULLS</span><span class="p">)</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">build</span><span class="p">()</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;last_wo_null&quot;</span><span class="p">),</span>
-   <span class="o">....</span><span class="p">:</span>     <span class="n">f</span><span class="o">.</span><span class="n">window</span><span class="p">(</span><span class="s2">&quot;last_value&quot;</span><span class="p">,</span> <span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Type 2&quot;&#39;</span><span class="p">)])</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">window_frame</span><span class="p">(</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">&quot;rows&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="mi">0</span><span class="p">))</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Speed&quot;&#39;</span><span class="p">))</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">null_treatment</span><span class="p">(</span><span class="n">NullTreatment</span><span class="o">.</span><span class="n">RESPECT_NULLS</span><span class="p">)</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">build</span><span class="p">()</span>
-   <span class="o">....</span><span class="p">:</span>         <span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;last_with_null&quot;</span><span class="p">)</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="n">f</span><span class="o">.</span><span class="n">last_value</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Type 2&quot;&#39;</span><span class="p">))</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="o">.</span><span class="n">over</span><span class="p">(</span>
+   <span class="o">....</span><span class="p">:</span>         <span class="n">Window</span><span class="p">(</span>
+   <span class="o">....</span><span class="p">:</span>             <span class="n">window_frame</span><span class="o">=</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">&quot;rows&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span>
+   <span class="o">....</span><span class="p">:</span>             <span class="n">order_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Speed&quot;&#39;</span><span class="p">)],</span>
+   <span class="o">....</span><span class="p">:</span>             <span class="n">null_treatment</span><span class="o">=</span><span class="n">NullTreatment</span><span class="o">.</span><span class="n">IGNORE_NULLS</span><span class="p">,</span>
+   <span class="o">....</span><span class="p">:</span>         <span class="p">)</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="p">)</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;last_wo_null&quot;</span><span class="p">),</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="n">f</span><span class="o">.</span><span class="n">last_value</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Type 2&quot;&#39;</span><span class="p">))</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="o">.</span><span class="n">over</span><span class="p">(</span>
+   <span class="o">....</span><span class="p">:</span>         <span class="n">Window</span><span class="p">(</span>
+   <span class="o">....</span><span class="p">:</span>             <span class="n">window_frame</span><span class="o">=</span><span class="n">WindowFrame</span><span class="p">(</span><span class="s2">&quot;rows&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span>
+   <span class="o">....</span><span class="p">:</span>             <span class="n">order_by</span><span class="o">=</span><span class="p">[</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;&quot;Speed&quot;&#39;</span><span class="p">)],</span>
+   <span class="o">....</span><span class="p">:</span>             <span class="n">null_treatment</span><span class="o">=</span><span class="n">NullTreatment</span><span class="o">.</span><span class="n">RESPECT_NULLS</span><span class="p">,</span>
+   <span class="o">....</span><span class="p">:</span>         <span class="p">)</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="p">)</span>
+   <span class="o">....</span><span class="p">:</span>     <span class="o">.</span><span class="n">alias</span><span class="p">(</span><span class="s2">&quot;last_with_null&quot;</span><span class="p">),</span>
    <span class="o">....</span><span class="p">:</span> <span class="p">)</span>
    <span class="o">....</span><span class="p">:</span> 
 <span class="n">Out</span><span class="p">[</span><span class="mi">12</span><span class="p">]:</span> 
diff --git a/user-guide/configuration.html b/user-guide/configuration.html
index f62b79b3..87ebc3ee 100644
--- a/user-guide/configuration.html
+++ b/user-guide/configuration.html
@@ -401,9 +401,33 @@
               
               <div class="toc-item">
                 
+<div class="tocsection onthispage pt-5 pb-3">
+    <i class="fas fa-list"></i> On this page
+</div>
 
 <nav id="bd-toc-nav">
-    
+    <ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+  <a class="reference internal nav-link" href="#maximizing-cpu-usage">
+   Maximizing CPU Usage
+  </a>
+  <ul class="visible nav section-nav flex-column">
+   <li class="toc-h3 nav-item toc-entry">
+    <a class="reference internal nav-link" href="#benchmark-example">
+     Benchmark Example
+    </a>
+    <ul class="nav section-nav flex-column">
+     <li class="toc-h4 nav-item toc-entry">
+      <a class="reference internal nav-link" href="#important-considerations">
+       Important Considerations
+      </a>
+     </li>
+    </ul>
+   </li>
+  </ul>
+ </li>
+</ul>
+
 </nav>
               </div>
               
@@ -451,8 +475,126 @@ a <a class="reference internal" href="../autoapi/datafusion/context/index.html#d
 <span class="nb">print</span><span class="p">(</span><span class="n">ctx</span><span class="p">)</span>
 </pre></div>
 </div>
-<p>You can read more about available <a class="reference internal" href="../autoapi/datafusion/context/index.html#datafusion.context.SessionConfig" title="datafusion.context.SessionConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">SessionConfig</span></code></a> options in the <a class="reference external" href="https://arrow.apache.org/datafusion/user-guide/configs.html">rust DataFusion Configuration guide</a>,
+<section id="maximizing-cpu-usage">
+<h2>Maximizing CPU Usage<a class="headerlink" href="#maximizing-cpu-usage" title="Link to this heading">¶</a></h2>
+<p>DataFusion uses partitions to parallelize work. For small queries the
+default configuration (number of CPU cores) is often sufficient, but to
+fully utilize available hardware you can tune how many partitions are
+created and when DataFusion will repartition data automatically.</p>
+<p>Configure a <code class="docutils literal notranslate"><span class="pre">SessionContext</span></code> with a higher partition count:</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionConfig</span><span class="p">,</span> <span class="n">SessionContext</span>
+
+<span class="c1"># allow up to 16 concurrent partitions</span>
+<span class="n">config</span> <span class="o">=</span> <span class="n">SessionConfig</span><span class="p">()</span><span class="o">.</span><span class="n">with_target_partitions</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span>
+<span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">(</span><span class="n">config</span><span class="p">)</span>
+</pre></div>
+</div>
+<p>Automatic repartitioning for joins, aggregations, window functions and
+other operations can be enabled to increase parallelism:</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">config</span> <span class="o">=</span> <span class="p">(</span>
+    <span class="n">SessionConfig</span><span class="p">()</span>
+    <span class="o">.</span><span class="n">with_target_partitions</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span>
+    <span class="o">.</span><span class="n">with_repartition_joins</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
+    <span class="o">.</span><span class="n">with_repartition_aggregations</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
+    <span class="o">.</span><span class="n">with_repartition_windows</span><span class="p">(</span><span class="kc">True</span><span class="p">)</span>
+<span class="p">)</span>
+</pre></div>
+</div>
+<p>Manual repartitioning is available on DataFrames when you need precise
+control:</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span>
+
+<span class="n">df</span> <span class="o">=</span> <span class="n">ctx</span><span class="o">.</span><span class="n">read_parquet</span><span class="p">(</span><span class="s2">&quot;data.parquet&quot;</span><span class="p">)</span>
+
+<span class="c1"># Evenly divide into 16 partitions</span>
+<span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">repartition</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span>
+
+<span class="c1"># Or partition by the hash of a column</span>
+<span class="n">df</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">repartition_by_hash</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;a&quot;</span><span class="p">),</span> <span class="n">num</span><span class="o">=</span><span class="mi">16</span><span class="p">)</span>
+
+<span class="n">result</span> <span class="o">=</span> <span class="n">df</span><span class="o">.</span><span class="n">collect</span><span class="p">()</span>
+</pre></div>
+</div>
+<section id="benchmark-example">
+<h3>Benchmark Example<a class="headerlink" href="#benchmark-example" title="Link to this heading">¶</a></h3>
+<p>The repository includes a benchmark script that demonstrates how to maximize CPU usage
+with DataFusion. The <code class="code docutils literal notranslate"><span class="pre">benchmarks/max_cpu_usage.py</span></code> script shows a practical example
+of configuring DataFusion for optimal parallelism.</p>
+<p>You can run the benchmark script to see the impact of different configuration settings:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># Run with default settings (uses all CPU cores)</span>
+python<span class="w"> </span>benchmarks/max_cpu_usage.py
+
+<span class="c1"># Run with specific number of rows and partitions</span>
+python<span class="w"> </span>benchmarks/max_cpu_usage.py<span class="w"> </span>--rows<span class="w"> </span><span class="m">5000000</span><span class="w"> </span>--partitions<span class="w"> </span><span class="m">16</span>
+
+<span class="c1"># See all available options</span>
+python<span class="w"> </span>benchmarks/max_cpu_usage.py<span class="w"> </span>--help
+</pre></div>
+</div>
+<p>Here’s an example showing the performance difference between single and multiple partitions:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># Single partition - slower processing</span>
+$<span class="w"> </span>python<span class="w"> </span>benchmarks/max_cpu_usage.py<span class="w"> </span>--rows<span class="o">=</span><span class="m">10000000</span><span class="w"> </span>--partitions<span class="w"> </span><span class="m">1</span>
+Processed<span class="w"> </span><span class="m">10000000</span><span class="w"> </span>rows<span class="w"> </span>using<span class="w"> </span><span class="m">1</span><span class="w"> </span>partitions<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">0</span>.107s
+
+<span class="c1"># Multiple partitions - faster processing</span>
+$<span class="w"> </span>python<span class="w"> </span>benchmarks/max_cpu_usage.py<span class="w"> </span>--rows<span class="o">=</span><span class="m">10000000</span><span class="w"> </span>--partitions<span class="w"> </span><span class="m">10</span>
+Processed<span class="w"> </span><span class="m">10000000</span><span class="w"> </span>rows<span class="w"> </span>using<span class="w"> </span><span class="m">10</span><span class="w"> </span>partitions<span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">0</span>.038s
+</pre></div>
+</div>
+<p>This example demonstrates nearly 3x performance improvement (0.107s vs 0.038s) when using
+10 partitions instead of 1, showcasing how proper partitioning can significantly improve
+CPU utilization and query performance.</p>
+<p>The script demonstrates several key optimization techniques:</p>
+<ol class="arabic simple">
+<li><p><strong>Higher target partition count</strong>: Uses <code class="code docutils literal notranslate"><span class="pre">with_target_partitions()</span></code> to set the number of concurrent partitions</p></li>
+<li><p><strong>Automatic repartitioning</strong>: Enables repartitioning for joins, aggregations, and window functions</p></li>
+<li><p><strong>Manual repartitioning</strong>: Uses <code class="code docutils literal notranslate"><span class="pre">repartition()</span></code> to ensure all partitions are utilized</p></li>
+<li><p><strong>CPU-intensive operations</strong>: Performs aggregations that can benefit from parallelization</p></li>
+</ol>
+<p>The benchmark creates synthetic data and measures the time taken to perform a sum aggregation
+across the specified number of partitions. This helps you understand how partition configuration
+affects performance on your specific hardware.</p>
+<section id="important-considerations">
+<h4>Important Considerations<a class="headerlink" href="#important-considerations" title="Link to this heading">¶</a></h4>
+<p>The provided benchmark script demonstrates partitioning concepts using synthetic in-memory data
+and simple aggregation operations. While useful for understanding basic configuration principles,
+actual performance in production environments may vary significantly based on numerous factors:</p>
+<p><strong>Data Sources and I/O Characteristics:</strong></p>
+<ul class="simple">
+<li><p><strong>Table providers</strong>: Performance differs greatly between Parquet files, CSV files, databases, and cloud storage</p></li>
+<li><p><strong>Storage type</strong>: Local SSD, network-attached storage, and cloud storage have vastly different characteristics</p></li>
+<li><p><strong>Network latency</strong>: Remote data sources introduce additional latency considerations</p></li>
+<li><p><strong>File sizes and distribution</strong>: Large files may benefit differently from partitioning than many small files</p></li>
+</ul>
+<p><strong>Query and Workload Characteristics:</strong></p>
+<ul class="simple">
+<li><p><strong>Operation complexity</strong>: Simple aggregations versus complex joins, window functions, or nested queries</p></li>
+<li><p><strong>Data distribution</strong>: Skewed data may not partition evenly, affecting parallel efficiency</p></li>
+<li><p><strong>Memory usage</strong>: Large datasets may require different memory management strategies</p></li>
+<li><p><strong>Concurrent workloads</strong>: Multiple queries running simultaneously affect resource allocation</p></li>
+</ul>
+<p><strong>Hardware and Environment Factors:</strong></p>
+<ul class="simple">
+<li><p><strong>CPU architecture</strong>: Different processors have varying parallel processing capabilities</p></li>
+<li><p><strong>Available memory</strong>: Limited RAM may require different optimization strategies</p></li>
+<li><p><strong>System load</strong>: Other applications competing for resources affect DataFusion performance</p></li>
+</ul>
+<p><strong>Recommendations for Production Use:</strong></p>
+<p>To optimize DataFusion for your specific use case, it is strongly recommended to:</p>
+<ol class="arabic simple">
+<li><p><strong>Create custom benchmarks</strong> using your actual data sources, formats, and query patterns</p></li>
+<li><p><strong>Test with representative data volumes</strong> that match your production workloads</p></li>
+<li><p><strong>Measure end-to-end performance</strong> including data loading, processing, and result handling</p></li>
+<li><p><strong>Evaluate different configuration combinations</strong> for your specific hardware and workload</p></li>
+<li><p><strong>Monitor resource utilization</strong> (CPU, memory, I/O) to identify bottlenecks in your environment</p></li>
+</ol>
+<p>This approach will provide more accurate insights into how DataFusion configuration options
+will impact your particular applications and infrastructure.</p>
+<p>For more information about available <a class="reference internal" href="../autoapi/datafusion/context/index.html#datafusion.context.SessionConfig" title="datafusion.context.SessionConfig"><code class="xref py py-class docutils literal notranslate"><span class="pre">SessionConfig</span></code></a> options, see the <a class="reference external" href="https://arrow.apache.org/datafusion/user-guide/configs.html">rust DataFusion Configuration guide</a>,
 and about <code class="code docutils literal notranslate"><span class="pre">RuntimeEnvBuilder</span></code> options in the rust <a class="reference external" href="https://docs.rs/datafusion/latest/datafusion/execution/runtime_env/struct.RuntimeEnvBuilder.html">online API documentation</a>.</p>
+</section>
+</section>
+</section>
 </section>
 
 
diff --git a/user-guide/data-sources.html b/user-guide/data-sources.html
index a2a08fc2..a97fd027 100644
--- a/user-guide/data-sources.html
+++ b/user-guide/data-sources.html
@@ -438,8 +438,8 @@
     </a>
    </li>
    <li class="toc-h2 nav-item toc-entry">
-    <a class="reference internal nav-link" href="#iceberg">
-     Iceberg
+    <a class="reference internal nav-link" href="#apache-iceberg">
+     Apache Iceberg
     </a>
    </li>
    <li class="toc-h2 nav-item toc-entry">
@@ -487,7 +487,7 @@
 <p>DataFusion provides a wide variety of ways to get data into a DataFrame to perform operations.</p>
 <section id="local-file">
 <h2>Local file<a class="headerlink" href="#local-file" title="Link to this heading">¶</a></h2>
-<p>DataFusion has the abilty to read from a variety of popular file formats, such as <a class="reference internal" href="io/parquet.html#io-parquet"><span class="std std-ref">Parquet</span></a>,
+<p>DataFusion has the ability to read from a variety of popular file formats, such as <a class="reference internal" href="io/parquet.html#io-parquet"><span class="std std-ref">Parquet</span></a>,
 <a class="reference internal" href="io/csv.html#io-csv"><span class="std std-ref">CSV</span></a>, <a class="reference internal" href="io/json.html#io-json"><span class="std std-ref">JSON</span></a>, and <a class="reference internal" href="io/avro.html#io-avro"><span class="std std-ref">AVRO</span></a>.</p>
 <div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span>
 
@@ -628,7 +628,7 @@ The example below requires an appropriate S3 account with access credentials.</p
 <a class="reference external" href="https://pola.rs/">Polars</a> and <a class="reference external" href="https://pandas.pydata.org/">Pandas</a>.
 Since DataFusion version 42.0.0, any DataFrame library that supports the Arrow FFI PyCapsule
 interface can be imported to DataFusion using the
-<a class="reference internal" href="../autoapi/datafusion/context/index.html#datafusion.context.SessionContext.from_arrow" title="datafusion.context.SessionContext.from_arrow"><code class="xref py py-func docutils literal notranslate"><span class="pre">from_arrow()</span></code></a> function. Older verions of Polars may
+<a class="reference internal" href="../autoapi/datafusion/context/index.html#datafusion.context.SessionContext.from_arrow" title="datafusion.context.SessionContext.from_arrow"><code class="xref py py-func docutils literal notranslate"><span class="pre">from_arrow()</span></code></a> function. Older versions of Polars may
 not support the arrow interface. In those cases, you can still import via the
 <a class="reference internal" href="../autoapi/datafusion/context/index.html#datafusion.context.SessionContext.from_polars" title="datafusion.context.SessionContext.from_polars"><code class="xref py py-func docutils literal notranslate"><span class="pre">from_polars()</span></code></a> function.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">pandas</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pd</span>
@@ -674,16 +674,44 @@ which can lead to a significant performance difference.</p>
 </pre></div>
 </div>
 </section>
-<section id="iceberg">
-<h2>Iceberg<a class="headerlink" href="#iceberg" title="Link to this heading">¶</a></h2>
-<p>Coming soon!</p>
+<section id="apache-iceberg">
+<h2>Apache Iceberg<a class="headerlink" href="#apache-iceberg" title="Link to this heading">¶</a></h2>
+<p>DataFusion 45.0.0 and later support the ability to register Apache Iceberg tables as table providers through the Custom Table Provider interface.</p>
+<p>This requires either the <a class="reference external" href="https://pypi.org/project/pyiceberg/">pyiceberg</a> library (&gt;=0.10.0) or the <a class="reference external" href="https://pypi.org/project/pyiceberg-core/">pyiceberg-core</a> library (&gt;=0.5.0).</p>
+<ul class="simple">
+<li><p>The <code class="docutils literal notranslate"><span class="pre">pyiceberg-core</span></code> library exposes Iceberg Rust’s implementation of the Custom Table Provider interface as python bindings.</p></li>
+<li><p>The <code class="docutils literal notranslate"><span class="pre">pyiceberg</span></code> library utilizes the <code class="docutils literal notranslate"><span class="pre">pyiceberg-core</span></code> python bindings under the hood and provides a native way for Python users to interact with the DataFusion.</p></li>
+</ul>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span>
+<span class="kn">from</span><span class="w"> </span><span class="nn">pyiceberg.catalog</span><span class="w"> </span><span class="kn">import</span> <span class="n">load_catalog</span>
+<span class="kn">import</span><span class="w"> </span><span class="nn">pyarrow</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">pa</span>
+
+<span class="c1"># Load catalog and create/load a table</span>
+<span class="n">catalog</span> <span class="o">=</span> <span class="n">load_catalog</span><span class="p">(</span><span class="s2">&quot;catalog&quot;</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="s2">&quot;in-memory&quot;</span><span class="p">)</span>
+<span class="n">catalog</span><span class="o">.</span><span class="n">create_namespace_if_not_exists</span><span class="p">(</span><span class="s2">&quot;default&quot;</span><span class="p">)</span>
+
+<span class="c1"># Create some sample data</span>
+<span class="n">data</span> <span class="o">=</span> <span class="n">pa</span><span class="o">.</span><span class="n">table</span><span class="p">({</span><span class="s2">&quot;x&quot;</span><span class="p">:</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="s2">&quot;y&quot;</span><span class="p">:</span> <span class="p">[</span><span class="m [...]
+<span class="n">iceberg_table</span> <span class="o">=</span> <span class="n">catalog</span><span class="o">.</span><span class="n">create_table</span><span class="p">(</span><span class="s2">&quot;default.test&quot;</span><span class="p">,</span> <span class="n">schema</span><span class="o">=</span><span class="n">data</span><span class="o">.</span><span class="n">schema</span><span class="p">)</span>
+<span class="n">iceberg_table</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
+
+<span class="c1"># Register the table with DataFusion</span>
+<span class="n">ctx</span> <span class="o">=</span> <span class="n">SessionContext</span><span class="p">()</span>
+<span class="n">ctx</span><span class="o">.</span><span class="n">register_table_provider</span><span class="p">(</span><span class="s2">&quot;test&quot;</span><span class="p">,</span> <span class="n">iceberg_table</span><span class="p">)</span>
+
+<span class="c1"># Query the table using DataFusion</span>
+<span class="n">ctx</span><span class="o">.</span><span class="n">table</span><span class="p">(</span><span class="s2">&quot;test&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
+</pre></div>
+</div>
+<p>Note that the Datafusion integration rely on features from the <a class="reference external" href="https://github.com/apache/iceberg-rust/">Iceberg Rust</a> implementation instead of the <a class="reference external" href="https://github.com/apache/iceberg-python/">PyIceberg</a> implementation.
+Features that are available in PyIceberg but not yet in Iceberg Rust will not be available when using DataFusion.</p>
 </section>
 <section id="custom-table-provider">
 <h2>Custom Table Provider<a class="headerlink" href="#custom-table-provider" title="Link to this heading">¶</a></h2>
 <p>You can implement a custom Data Provider in Rust and expose it to DataFusion through the
 the interface as describe in the <a class="reference internal" href="io/table_provider.html#io-custom-table-provider"><span class="std std-ref">Custom Table Provider</span></a>
 section. This is an advanced topic, but a
-<a class="reference external" href="https://github.com/apache/datafusion-python/tree/main/examples/ffi-table-provider">user example</a>
+<a class="reference external" href="https://github.com/apache/datafusion-python/tree/main/examples/datafusion-ffi-example">user example</a>
 is provided in the DataFusion repository.</p>
 </section>
 </section>
diff --git a/user-guide/dataframe/index.html b/user-guide/dataframe/index.html
index c463e85c..cf5c2acd 100644
--- a/user-guide/dataframe/index.html
+++ b/user-guide/dataframe/index.html
@@ -422,6 +422,11 @@
    Common DataFrame Operations
   </a>
  </li>
+ <li class="toc-h2 nav-item toc-entry">
+  <a class="reference internal nav-link" href="#column-names-as-function-arguments">
+   Column Names as Function Arguments
+  </a>
+ </li>
  <li class="toc-h2 nav-item toc-entry">
   <a class="reference internal nav-link" href="#terminal-operations">
    Terminal Operations
@@ -571,6 +576,46 @@ For custom data sources, see <a class="reference internal" href="../io/table_pro
 </pre></div>
 </div>
 </section>
+<section id="column-names-as-function-arguments">
+<h2>Column Names as Function Arguments<a class="headerlink" href="#column-names-as-function-arguments" title="Link to this heading">¶</a></h2>
+<p>Some <code class="docutils literal notranslate"><span class="pre">DataFrame</span></code> methods accept column names when an argument refers to an
+existing column. These include:</p>
+<ul class="simple">
+<li><p><code class="xref py py-meth docutils literal notranslate"><span class="pre">select()</span></code></p></li>
+<li><p><code class="xref py py-meth docutils literal notranslate"><span class="pre">sort()</span></code></p></li>
+<li><p><code class="xref py py-meth docutils literal notranslate"><span class="pre">drop()</span></code></p></li>
+<li><p><code class="xref py py-meth docutils literal notranslate"><span class="pre">join()</span></code> (<code class="docutils literal notranslate"><span class="pre">on</span></code> argument)</p></li>
+<li><p><code class="xref py py-meth docutils literal notranslate"><span class="pre">aggregate()</span></code> (grouping columns)</p></li>
+</ul>
+<p>See the full function documentation for details on any specific function.</p>
+<p>Note that <code class="xref py py-meth docutils literal notranslate"><span class="pre">join_on()</span></code> expects <code class="docutils literal notranslate"><span class="pre">col()</span></code>/<code class="docutils literal notranslate"><span class="pre">column()</span></code> expressions rather than plain strings.</p>
+<p>For such methods, you can pass column names directly:</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span><span class="p">,</span> <span class="n">functions</span> <span class="k">as</span> <span class="n">f</span>
+
+<span class="n">df</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="s1">&#39;id&#39;</span><span class="p">)</span>
+<span class="n">df</span><span class="o">.</span><span class="n">aggregate</span><span class="p">(</span><span class="s1">&#39;id&#39;</span><span class="p">,</span> <span class="p">[</span><span class="n">f</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;value&#39;</span><span class="p">))])</span>
+</pre></div>
+</div>
+<p>The same operation can also be written with explicit column expressions, using either <code class="docutils literal notranslate"><span class="pre">col()</span></code> or <code class="docutils literal notranslate"><span class="pre">column()</span></code>:</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span><span class="p">,</span> <span class="n">column</span><span class="p">,</span> <span class="n">functions</span> <span class="k">as</span> <span class="n">f</span>
+
+<span class="n">df</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;id&#39;</span><span class="p">))</span>
+<span class="n">df</span><span class="o">.</span><span class="n">aggregate</span><span class="p">(</span><span class="n">column</span><span class="p">(</span><span class="s1">&#39;id&#39;</span><span class="p">),</span> <span class="p">[</span><span class="n">f</span><span class="o">.</span><span class="n">count</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s1">&#39;value&#39;</span><span class="p">))])</span>
+</pre></div>
+</div>
+<p>Note that <code class="docutils literal notranslate"><span class="pre">column()</span></code> is an alias of <code class="docutils literal notranslate"><span class="pre">col()</span></code>, so you can use either name; the example above shows both in action.</p>
+<p>Whenever an argument represents an expression—such as in
+<code class="xref py py-meth docutils literal notranslate"><span class="pre">filter()</span></code> or
+<code class="xref py py-meth docutils literal notranslate"><span class="pre">with_column()</span></code>—use <code class="docutils literal notranslate"><span class="pre">col()</span></code> to reference
+columns. The comparison and arithmetic operators on <code class="docutils literal notranslate"><span class="pre">Expr</span></code> will automatically
+convert any non-<code class="docutils literal notranslate"><span class="pre">Expr</span></code> value into a literal expression, so writing</p>
+<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">col</span>
+<span class="n">df</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">col</span><span class="p">(</span><span class="s2">&quot;age&quot;</span><span class="p">)</span> <span class="o">&gt;</span> <span class="mi">21</span><span class="p">)</span>
+</pre></div>
+</div>
+<p>is equivalent to using <code class="docutils literal notranslate"><span class="pre">lit(21)</span></code> explicitly. Use <code class="docutils literal notranslate"><span class="pre">lit()</span></code> (also available
+as <code class="docutils literal notranslate"><span class="pre">literal()</span></code>) when you need to construct a literal expression directly.</p>
+</section>
 <section id="terminal-operations">
 <h2>Terminal Operations<a class="headerlink" href="#terminal-operations" title="Link to this heading">¶</a></h2>
 <p>To materialize the results of your DataFrame operations:</p>
diff --git a/user-guide/introduction.html b/user-guide/introduction.html
index e04de54e..df9a43fa 100644
--- a/user-guide/introduction.html
+++ b/user-guide/introduction.html
@@ -447,7 +447,7 @@ DataFusion through various examples and highlight the most effective ways of usi
 <div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">1</span><span class="p">]:</span> <span class="kn">import</span><span class="w"> </span><span class="nn">datafusion</span>
 
 <span class="n">In</span> <span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="n">datafusion</span><span class="o">.</span><span class="n">__version__</span>
-<span class="n">Out</span><span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="s1">&#39;49.0.0&#39;</span>
+<span class="n">Out</span><span class="p">[</span><span class="mi">2</span><span class="p">]:</span> <span class="s1">&#39;50.0.0&#39;</span>
 </pre></div>
 </div>
 <p>In this documentation we will also show some examples for how DataFusion integrates


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org