You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/05/20 23:34:22 UTC
(datafusion-python) branch asf-staging updated: Publish built docs triggered by dac9ec6230dba8717d8a0d27de19a141600486b1
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/datafusion-python.git
The following commit(s) were added to refs/heads/asf-staging by this push:
new 1f1daeeb Publish built docs triggered by dac9ec6230dba8717d8a0d27de19a141600486b1
1f1daeeb is described below
commit 1f1daeeb5e237d1b967ac1f4f9783699af671d44
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Wed May 20 23:34:18 2026 +0000
Publish built docs triggered by dac9ec6230dba8717d8a0d27de19a141600486b1
---
_sources/autoapi/datafusion/expr/index.rst.txt | 25 +++++----
_sources/autoapi/datafusion/index.rst.txt | 47 ++++++++++++----
_sources/autoapi/datafusion/ipc/index.rst.txt | 19 ++++---
.../autoapi/datafusion/user_defined/index.rst.txt | 22 ++++++++
.../common-operations/aggregations.rst.txt | 18 ++++++
.../user-guide/common-operations/windows.rst.txt | 18 ++++++
autoapi/datafusion/expr/index.html | 25 +++++----
autoapi/datafusion/index.html | 61 +++++++++++++++++----
autoapi/datafusion/ipc/index.html | 19 ++++---
autoapi/datafusion/user_defined/index.html | 36 ++++++++++++
genindex.html | 14 ++++-
objects.inv | Bin 10667 -> 10691 bytes
searchindex.js | 2 +-
user-guide/common-operations/aggregations.html | 49 ++++++++++++-----
user-guide/common-operations/basic-info.html | 34 ++++++------
user-guide/common-operations/functions.html | 20 +++----
user-guide/common-operations/index.html | 2 +
user-guide/common-operations/udf-and-udfa.html | 4 +-
user-guide/common-operations/windows.html | 23 ++++++++
19 files changed, 326 insertions(+), 112 deletions(-)
diff --git a/_sources/autoapi/datafusion/expr/index.rst.txt b/_sources/autoapi/datafusion/expr/index.rst.txt
index da1fc67c..21702da8 100644
--- a/_sources/autoapi/datafusion/expr/index.rst.txt
+++ b/_sources/autoapi/datafusion/expr/index.rst.txt
@@ -320,12 +320,13 @@ Module Contents
Lets expressions be shipped to worker processes via
:func:`pickle.dumps` / :func:`pickle.loads`. Built-in functions
- and Python scalar UDFs travel inside the pickle bytes; only
- FFI-capsule UDFs require pre-registration on the worker. The
- worker's :class:`SessionContext` for resolving those references
- is looked up via :func:`datafusion.ipc.set_worker_ctx`, falling
- back to the global :class:`SessionContext` if none has been
- installed on the worker.
+ and Python UDFs (scalar, aggregate, window) travel inside the
+ pickle bytes; only FFI-capsule UDFs require pre-registration on
+ the worker. The worker's :class:`SessionContext` for resolving
+ those references is looked up via
+ :func:`datafusion.ipc.set_worker_ctx`, falling back to the
+ global :class:`SessionContext` if none has been installed on
+ the worker.
.. warning:: Security
:func:`pickle.loads` on the returned tuple executes
@@ -691,7 +692,7 @@ Module Contents
``ctx`` is ``None`` the worker context installed via
:func:`datafusion.ipc.set_worker_ctx` is consulted; if no worker
context is installed, the global :class:`SessionContext` is used
- (sufficient for built-ins and Python scalar UDFs, plus any UDFs
+ (sufficient for built-ins and Python UDFs, plus any UDFs
registered on the global context).
.. warning:: Security
@@ -1058,14 +1059,14 @@ Module Contents
installed :class:`LogicalExtensionCodec`. When ``ctx`` is
``None``, the default codec is used.
- Built-in functions and Python scalar UDFs travel inside the
- returned bytes; the worker does not need to pre-register them.
- UDFs imported via the FFI capsule protocol travel by name only
- and must be registered on the worker.
+ Built-in functions and Python UDFs (scalar, aggregate, window)
+ travel inside the returned bytes; the worker does not need to
+ pre-register them. UDFs imported via the FFI capsule protocol
+ travel by name only and must be registered on the worker.
.. warning:: Security
Bytes returned here may embed a cloudpickled Python
- callable (when the expression carries a Python scalar UDF).
+ callable (when the expression carries a Python UDF).
Reconstructing them via :meth:`from_bytes` or
:func:`pickle.loads` executes arbitrary Python on the
receiver. Only accept payloads from trusted sources.
diff --git a/_sources/autoapi/datafusion/index.rst.txt b/_sources/autoapi/datafusion/index.rst.txt
index 3bc8e816..dc78e4ba 100644
--- a/_sources/autoapi/datafusion/index.rst.txt
+++ b/_sources/autoapi/datafusion/index.rst.txt
@@ -331,6 +331,17 @@ Package Contents
.. py:attribute:: _udaf
+ .. py:property:: name
+ :type: str
+
+
+ Return the registered name of this UDAF.
+
+ For UDAFs imported via the FFI capsule protocol, this is the
+ name the capsule itself reports — not the ``name`` argument
+ passed to the constructor (which is ignored on the FFI path).
+
+
.. py:class:: Catalog(catalog: datafusion._internal.catalog.RawCatalog)
DataFusion data catalog.
@@ -895,12 +906,13 @@ Package Contents
Lets expressions be shipped to worker processes via
:func:`pickle.dumps` / :func:`pickle.loads`. Built-in functions
- and Python scalar UDFs travel inside the pickle bytes; only
- FFI-capsule UDFs require pre-registration on the worker. The
- worker's :class:`SessionContext` for resolving those references
- is looked up via :func:`datafusion.ipc.set_worker_ctx`, falling
- back to the global :class:`SessionContext` if none has been
- installed on the worker.
+ and Python UDFs (scalar, aggregate, window) travel inside the
+ pickle bytes; only FFI-capsule UDFs require pre-registration on
+ the worker. The worker's :class:`SessionContext` for resolving
+ those references is looked up via
+ :func:`datafusion.ipc.set_worker_ctx`, falling back to the
+ global :class:`SessionContext` if none has been installed on
+ the worker.
.. warning:: Security
:func:`pickle.loads` on the returned tuple executes
@@ -1266,7 +1278,7 @@ Package Contents
``ctx`` is ``None`` the worker context installed via
:func:`datafusion.ipc.set_worker_ctx` is consulted; if no worker
context is installed, the global :class:`SessionContext` is used
- (sufficient for built-ins and Python scalar UDFs, plus any UDFs
+ (sufficient for built-ins and Python UDFs, plus any UDFs
registered on the global context).
.. warning:: Security
@@ -1633,14 +1645,14 @@ Package Contents
installed :class:`LogicalExtensionCodec`. When ``ctx`` is
``None``, the default codec is used.
- Built-in functions and Python scalar UDFs travel inside the
- returned bytes; the worker does not need to pre-register them.
- UDFs imported via the FFI capsule protocol travel by name only
- and must be registered on the worker.
+ Built-in functions and Python UDFs (scalar, aggregate, window)
+ travel inside the returned bytes; the worker does not need to
+ pre-register them. UDFs imported via the FFI capsule protocol
+ travel by name only and must be registered on the worker.
.. warning:: Security
Bytes returned here may embed a cloudpickled Python
- callable (when the expression carries a Python scalar UDF).
+ callable (when the expression carries a Python UDF).
Reconstructing them via :meth:`from_bytes` or
:func:`pickle.loads` executes arbitrary Python on the
receiver. Only accept payloads from trusted sources.
@@ -3171,6 +3183,17 @@ Package Contents
.. py:attribute:: _udwf
+ .. py:property:: name
+ :type: str
+
+
+ Return the registered name of this UDWF.
+
+ For UDWFs imported via the FFI capsule protocol, this is the
+ name the capsule itself reports — not the ``name`` argument
+ passed to the constructor (which is ignored on the FFI path).
+
+
.. py:function:: configure_formatter(**kwargs: Any) -> None
Configure the global DataFrame HTML formatter.
diff --git a/_sources/autoapi/datafusion/ipc/index.rst.txt b/_sources/autoapi/datafusion/ipc/index.rst.txt
index a1e3ff7d..26fcc9eb 100644
--- a/_sources/autoapi/datafusion/ipc/index.rst.txt
+++ b/_sources/autoapi/datafusion/ipc/index.rst.txt
@@ -25,16 +25,17 @@ datafusion.ipc
ctx.register_udaf(my_ffi_aggregate)
set_worker_ctx(ctx)
- Built-in functions and Python scalar UDFs travel inside the shipped
- expression itself and do not need pre-registration on the worker.
+ Built-in functions and Python UDFs (scalar, aggregate, window) travel
+ inside the shipped expression itself and do not need pre-registration
+ on the worker.
.. note:: Serialization model
- Expressions containing Python scalar UDFs are serialized using
- :mod:`cloudpickle`. The callable itself travels **by value**
- (bytecode and closure cells inlined), but any names the callable
- resolves via ``import`` are captured **by reference** and must be
- importable on the receiving worker.
+ Expressions containing Python UDFs (scalar, aggregate, window) are
+ serialized using :mod:`cloudpickle`. The callable itself travels
+ **by value** (bytecode and closure cells inlined), but any names the
+ callable resolves via ``import`` are captured **by reference** and
+ must be importable on the receiving worker.
The serialized payload is stamped with the sender's Python
``(major, minor)`` version. Loading on a different minor version
@@ -64,8 +65,8 @@ Module Contents
After clearing, expressions reconstructed in this worker fall back to
the global :class:`SessionContext` — adequate for built-ins and Python
- scalar UDFs, but anything imported via the FFI capsule protocol must
- be registered on the global context to resolve.
+ UDFs (scalar, aggregate, window), but anything imported via the FFI
+ capsule protocol must be registered on the global context to resolve.
.. rubric:: Examples
diff --git a/_sources/autoapi/datafusion/user_defined/index.rst.txt b/_sources/autoapi/datafusion/user_defined/index.rst.txt
index fe381393..e12afdb3 100644
--- a/_sources/autoapi/datafusion/user_defined/index.rst.txt
+++ b/_sources/autoapi/datafusion/user_defined/index.rst.txt
@@ -248,6 +248,17 @@ Module Contents
.. py:attribute:: _udaf
+ .. py:property:: name
+ :type: str
+
+
+ Return the registered name of this UDAF.
+
+ For UDAFs imported via the FFI capsule protocol, this is the
+ name the capsule itself reports — not the ``name`` argument
+ passed to the constructor (which is ignored on the FFI path).
+
+
.. py:class:: AggregateUDFExportable
Bases: :py:obj:`Protocol`
@@ -828,6 +839,17 @@ Module Contents
.. py:attribute:: _udwf
+ .. py:property:: name
+ :type: str
+
+
+ Return the registered name of this UDWF.
+
+ For UDWFs imported via the FFI capsule protocol, this is the
+ name the capsule itself reports — not the ``name`` argument
+ passed to the constructor (which is ignored on the FFI path).
+
+
.. py:class:: WindowUDFExportable
Bases: :py:obj:`Protocol`
diff --git a/_sources/user-guide/common-operations/aggregations.rst.txt b/_sources/user-guide/common-operations/aggregations.rst.txt
index f59b62ab..8f218abd 100644
--- a/_sources/user-guide/common-operations/aggregations.rst.txt
+++ b/_sources/user-guide/common-operations/aggregations.rst.txt
@@ -434,3 +434,21 @@ The available aggregate functions are:
- :py:meth:`datafusion.expr.GroupingSet.cube`
- :py:meth:`datafusion.expr.GroupingSet.grouping_sets`
+User-Defined Aggregate Functions
+--------------------------------
+
+You can ship custom aggregations to the engine by subclassing
+:py:class:`~datafusion.user_defined.Accumulator` and registering it via
+:py:func:`~datafusion.udaf`. See :py:mod:`datafusion.user_defined` for
+the accumulator interface and worked examples.
+
+.. note:: Serialization
+
+ Python aggregate UDFs travel inline inside pickled or
+ :py:meth:`~datafusion.expr.Expr.to_bytes`-serialized expressions —
+ the accumulator class is captured by value via :mod:`cloudpickle`,
+ so worker processes do not need to pre-register the UDF. Any names
+ the accumulator resolves via ``import`` are captured **by reference**
+ and must be importable on the receiving worker. See
+ :py:mod:`datafusion.ipc` for the full IPC model and security caveats.
+
diff --git a/_sources/user-guide/common-operations/windows.rst.txt b/_sources/user-guide/common-operations/windows.rst.txt
index d77881bc..127f691b 100644
--- a/_sources/user-guide/common-operations/windows.rst.txt
+++ b/_sources/user-guide/common-operations/windows.rst.txt
@@ -213,3 +213,21 @@ The possible window functions are:
3. Aggregate Functions
- All :ref:`Aggregation Functions<aggregation>` can be used as window functions.
+
+User-Defined Window Functions
+-----------------------------
+
+You can ship custom window functions to the engine by subclassing
+:py:class:`~datafusion.user_defined.WindowEvaluator` and registering it
+via :py:func:`~datafusion.udwf`. See :py:mod:`datafusion.user_defined`
+for the evaluator interface and worked examples.
+
+.. note:: Serialization
+
+ Python window UDFs travel inline inside pickled or
+ :py:meth:`~datafusion.expr.Expr.to_bytes`-serialized expressions —
+ the evaluator class is captured by value via :mod:`cloudpickle`, so
+ worker processes do not need to pre-register the UDF. Any names the
+ evaluator resolves via ``import`` are captured **by reference** and
+ must be importable on the receiving worker. See
+ :py:mod:`datafusion.ipc` for the full IPC model and security caveats.
diff --git a/autoapi/datafusion/expr/index.html b/autoapi/datafusion/expr/index.html
index 143998c8..e0221349 100644
--- a/autoapi/datafusion/expr/index.html
+++ b/autoapi/datafusion/expr/index.html
@@ -3174,12 +3174,13 @@ where the index begins at <code class="docutils literal notranslate"><span class
<dd><p>Pickle protocol hook.</p>
<p>Lets expressions be shipped to worker processes via
<code class="xref py py-func docutils literal notranslate"><span class="pre">pickle.dumps()</span></code> / <code class="xref py py-func docutils literal notranslate"><span class="pre">pickle.loads()</span></code>. Built-in functions
-and Python scalar UDFs travel inside the pickle bytes; only
-FFI-capsule UDFs require pre-registration on the worker. The
-worker’s <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> for resolving those references
-is looked up via <a class="reference internal" href="../ipc/index.html#datafusion.ipc.set_worker_ctx" title="datafusion.ipc.set_worker_ctx"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.ipc.set_worker_ctx()</span></code></a>, falling
-back to the global <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> if none has been
-installed on the worker.</p>
+and Python UDFs (scalar, aggregate, window) travel inside the
+pickle bytes; only FFI-capsule UDFs require pre-registration on
+the worker. The worker’s <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> for resolving
+those references is looked up via
+<a class="reference internal" href="../ipc/index.html#datafusion.ipc.set_worker_ctx" title="datafusion.ipc.set_worker_ctx"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.ipc.set_worker_ctx()</span></code></a>, falling back to the
+global <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> if none has been installed on
+the worker.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Security
@@ -3553,7 +3554,7 @@ function references that travel by name (e.g. FFI UDFs). When
<code class="docutils literal notranslate"><span class="pre">ctx</span></code> is <code class="docutils literal notranslate"><span class="pre">None</span></code> the worker context installed via
<a class="reference internal" href="../ipc/index.html#datafusion.ipc.set_worker_ctx" title="datafusion.ipc.set_worker_ctx"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.ipc.set_worker_ctx()</span></code></a> is consulted; if no worker
context is installed, the global <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> is used
-(sufficient for built-ins and Python scalar UDFs, plus any UDFs
+(sufficient for built-ins and Python UDFs, plus any UDFs
registered on the global context).</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
@@ -3915,15 +3916,15 @@ worker process for distributed evaluation.</p>
<p>When <code class="docutils literal notranslate"><span class="pre">ctx</span></code> is supplied, encoding routes through that session’s
installed <code class="xref py py-class docutils literal notranslate"><span class="pre">LogicalExtensionCodec</span></code>. When <code class="docutils literal notranslate"><span class="pre">ctx</span></code> is
<code class="docutils literal notranslate"><span class="pre">None</span></code>, the default codec is used.</p>
-<p>Built-in functions and Python scalar UDFs travel inside the
-returned bytes; the worker does not need to pre-register them.
-UDFs imported via the FFI capsule protocol travel by name only
-and must be registered on the worker.</p>
+<p>Built-in functions and Python UDFs (scalar, aggregate, window)
+travel inside the returned bytes; the worker does not need to
+pre-register them. UDFs imported via the FFI capsule protocol
+travel by name only and must be registered on the worker.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Security
Bytes returned here may embed a cloudpickled Python
-callable (when the expression carries a Python scalar UDF).
+callable (when the expression carries a Python UDF).
Reconstructing them via <a class="reference internal" href="#datafusion.expr.Expr.from_bytes" title="datafusion.expr.Expr.from_bytes"><code class="xref py py-meth docutils literal notranslate"><span class="pre">from_bytes()</span></code></a> or
<code class="xref py py-func docutils literal notranslate"><span class="pre">pickle.loads()</span></code> executes arbitrary Python on the
receiver. Only accept payloads from trusted sources.</p>
diff --git a/autoapi/datafusion/index.html b/autoapi/datafusion/index.html
index 9a5f6f95..40088e64 100644
--- a/autoapi/datafusion/index.html
+++ b/autoapi/datafusion/index.html
@@ -563,6 +563,15 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.AggregateUDF.name">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ AggregateUDF.name
+ </span>
+ </code>
+ </a>
+ </li>
</ul>
</li>
<li class="toc-h3 nav-item toc-entry">
@@ -3608,6 +3617,15 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.WindowUDF.name">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ WindowUDF.name
+ </span>
+ </code>
+ </a>
+ </li>
</ul>
</li>
<li class="toc-h3 nav-item toc-entry">
@@ -4156,6 +4174,15 @@ aggregation or window function calls.</p>
<span class="sig-name descname"><span class="pre">_udaf</span></span><a class="headerlink" href="#datafusion.AggregateUDF._udaf" title="Link to this definition">¶</a></dt>
<dd></dd></dl>
+<dl class="py property">
+<dt class="sig sig-object py" id="datafusion.AggregateUDF.name">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">name</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">str</span></em><a class="headerlink" href="#datafusion.AggregateUDF.name" title="Link to this definition">¶</a></dt>
+<dd><p>Return the registered name of this UDAF.</p>
+<p>For UDAFs imported via the FFI capsule protocol, this is the
+name the capsule itself reports — not the <code class="docutils literal notranslate"><span class="pre">name</span></code> argument
+passed to the constructor (which is ignored on the FFI path).</p>
+</dd></dl>
+
</dd></dl>
<dl class="py class">
@@ -4701,12 +4728,13 @@ where the index begins at <code class="docutils literal notranslate"><span class
<dd><p>Pickle protocol hook.</p>
<p>Lets expressions be shipped to worker processes via
<code class="xref py py-func docutils literal notranslate"><span class="pre">pickle.dumps()</span></code> / <code class="xref py py-func docutils literal notranslate"><span class="pre">pickle.loads()</span></code>. Built-in functions
-and Python scalar UDFs travel inside the pickle bytes; only
-FFI-capsule UDFs require pre-registration on the worker. The
-worker’s <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> for resolving those references
-is looked up via <a class="reference internal" href="ipc/index.html#datafusion.ipc.set_worker_ctx" title="datafusion.ipc.set_worker_ctx"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.ipc.set_worker_ctx()</span></code></a>, falling
-back to the global <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> if none has been
-installed on the worker.</p>
+and Python UDFs (scalar, aggregate, window) travel inside the
+pickle bytes; only FFI-capsule UDFs require pre-registration on
+the worker. The worker’s <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> for resolving
+those references is looked up via
+<a class="reference internal" href="ipc/index.html#datafusion.ipc.set_worker_ctx" title="datafusion.ipc.set_worker_ctx"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.ipc.set_worker_ctx()</span></code></a>, falling back to the
+global <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> if none has been installed on
+the worker.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Security
@@ -5080,7 +5108,7 @@ function references that travel by name (e.g. FFI UDFs). When
<code class="docutils literal notranslate"><span class="pre">ctx</span></code> is <code class="docutils literal notranslate"><span class="pre">None</span></code> the worker context installed via
<a class="reference internal" href="ipc/index.html#datafusion.ipc.set_worker_ctx" title="datafusion.ipc.set_worker_ctx"><code class="xref py py-func docutils literal notranslate"><span class="pre">datafusion.ipc.set_worker_ctx()</span></code></a> is consulted; if no worker
context is installed, the global <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> is used
-(sufficient for built-ins and Python scalar UDFs, plus any UDFs
+(sufficient for built-ins and Python UDFs, plus any UDFs
registered on the global context).</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
@@ -5442,15 +5470,15 @@ worker process for distributed evaluation.</p>
<p>When <code class="docutils literal notranslate"><span class="pre">ctx</span></code> is supplied, encoding routes through that session’s
installed <code class="xref py py-class docutils literal notranslate"><span class="pre">LogicalExtensionCodec</span></code>. When <code class="docutils literal notranslate"><span class="pre">ctx</span></code> is
<code class="docutils literal notranslate"><span class="pre">None</span></code>, the default codec is used.</p>
-<p>Built-in functions and Python scalar UDFs travel inside the
-returned bytes; the worker does not need to pre-register them.
-UDFs imported via the FFI capsule protocol travel by name only
-and must be registered on the worker.</p>
+<p>Built-in functions and Python UDFs (scalar, aggregate, window)
+travel inside the returned bytes; the worker does not need to
+pre-register them. UDFs imported via the FFI capsule protocol
+travel by name only and must be registered on the worker.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>Security
Bytes returned here may embed a cloudpickled Python
-callable (when the expression carries a Python scalar UDF).
+callable (when the expression carries a Python UDF).
Reconstructing them via <a class="reference internal" href="#datafusion.Expr.from_bytes" title="datafusion.Expr.from_bytes"><code class="xref py py-meth docutils literal notranslate"><span class="pre">from_bytes()</span></code></a> or
<code class="xref py py-func docutils literal notranslate"><span class="pre">pickle.loads()</span></code> executes arbitrary Python on the
receiver. Only accept payloads from trusted sources.</p>
@@ -7075,6 +7103,15 @@ the online documentation for more information.</p></li>
<span class="sig-name descname"><span class="pre">_udwf</span></span><a class="headerlink" href="#datafusion.WindowUDF._udwf" title="Link to this definition">¶</a></dt>
<dd></dd></dl>
+<dl class="py property">
+<dt class="sig sig-object py" id="datafusion.WindowUDF.name">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">name</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">str</span></em><a class="headerlink" href="#datafusion.WindowUDF.name" title="Link to this definition">¶</a></dt>
+<dd><p>Return the registered name of this UDWF.</p>
+<p>For UDWFs imported via the FFI capsule protocol, this is the
+name the capsule itself reports — not the <code class="docutils literal notranslate"><span class="pre">name</span></code> argument
+passed to the constructor (which is ignored on the FFI path).</p>
+</dd></dl>
+
</dd></dl>
<dl class="py function">
diff --git a/autoapi/datafusion/ipc/index.html b/autoapi/datafusion/ipc/index.html
index ac769960..9b0d23b4 100644
--- a/autoapi/datafusion/ipc/index.html
+++ b/autoapi/datafusion/ipc/index.html
@@ -506,16 +506,17 @@ once per worker:</p>
<span class="n">set_worker_ctx</span><span class="p">(</span><span class="n">ctx</span><span class="p">)</span>
</pre></div>
</div>
-<p>Built-in functions and Python scalar UDFs travel inside the shipped
-expression itself and do not need pre-registration on the worker.</p>
+<p>Built-in functions and Python UDFs (scalar, aggregate, window) travel
+inside the shipped expression itself and do not need pre-registration
+on the worker.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Serialization model</p>
-<p>Expressions containing Python scalar UDFs are serialized using
-<code class="xref py py-mod docutils literal notranslate"><span class="pre">cloudpickle</span></code>. The callable itself travels <strong>by value</strong>
-(bytecode and closure cells inlined), but any names the callable
-resolves via <code class="docutils literal notranslate"><span class="pre">import</span></code> are captured <strong>by reference</strong> and must be
-importable on the receiving worker.</p>
+<p>Expressions containing Python UDFs (scalar, aggregate, window) are
+serialized using <code class="xref py py-mod docutils literal notranslate"><span class="pre">cloudpickle</span></code>. The callable itself travels
+<strong>by value</strong> (bytecode and closure cells inlined), but any names the
+callable resolves via <code class="docutils literal notranslate"><span class="pre">import</span></code> are captured <strong>by reference</strong> and
+must be importable on the receiving worker.</p>
<p>The serialized payload is stamped with the sender’s Python
<code class="docutils literal notranslate"><span class="pre">(major,</span> <span class="pre">minor)</span></code> version. Loading on a different minor version
raises <code class="xref py py-class docutils literal notranslate"><span class="pre">ValueError</span></code> with an actionable message — cloudpickle
@@ -547,8 +548,8 @@ value vs. by reference.</p>
<dd><p>Remove this worker’s installed <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code>.</p>
<p>After clearing, expressions reconstructed in this worker fall back to
the global <code class="xref py py-class docutils literal notranslate"><span class="pre">SessionContext</span></code> — adequate for built-ins and Python
-scalar UDFs, but anything imported via the FFI capsule protocol must
-be registered on the global context to resolve.</p>
+UDFs (scalar, aggregate, window), but anything imported via the FFI
+capsule protocol must be registered on the global context to resolve.</p>
<p class="rubric">Examples</p>
<div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion</span><span class="w"> </span><span class="kn">import</span> <span class="n">SessionContext</span>
<span class="gp">>>> </span><span class="kn">from</span><span class="w"> </span><span class="nn">datafusion.ipc</span><span class="w"> </span><span class="kn">import</span> <span class="n">set_worker_ctx</span><span class="p">,</span> <span class="n">clear_worker_ctx</span><span class="p">,</span> <span class="n">get_worker_ctx</span>
diff --git a/autoapi/datafusion/user_defined/index.html b/autoapi/datafusion/user_defined/index.html
index 2015a44a..4aa8a12d 100644
--- a/autoapi/datafusion/user_defined/index.html
+++ b/autoapi/datafusion/user_defined/index.html
@@ -547,6 +547,15 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.user_defined.AggregateUDF.name">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ AggregateUDF.name
+ </span>
+ </code>
+ </a>
+ </li>
</ul>
</li>
<li class="toc-h3 nav-item toc-entry">
@@ -948,6 +957,15 @@
</code>
</a>
</li>
+ <li class="toc-h4 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#datafusion.user_defined.WindowUDF.name">
+ <code class="docutils literal notranslate">
+ <span class="pre">
+ WindowUDF.name
+ </span>
+ </code>
+ </a>
+ </li>
</ul>
</li>
<li class="toc-h3 nav-item toc-entry">
@@ -1336,6 +1354,15 @@ aggregation or window function calls.</p>
<span class="sig-name descname"><span class="pre">_udaf</span></span><a class="headerlink" href="#datafusion.user_defined.AggregateUDF._udaf" title="Link to this definition">¶</a></dt>
<dd></dd></dl>
+<dl class="py property">
+<dt class="sig sig-object py" id="datafusion.user_defined.AggregateUDF.name">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">name</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">str</span></em><a class="headerlink" href="#datafusion.user_defined.AggregateUDF.name" title="Link to this definition">¶</a></dt>
+<dd><p>Return the registered name of this UDAF.</p>
+<p>For UDAFs imported via the FFI capsule protocol, this is the
+name the capsule itself reports — not the <code class="docutils literal notranslate"><span class="pre">name</span></code> argument
+passed to the constructor (which is ignored on the FFI path).</p>
+</dd></dl>
+
</dd></dl>
<dl class="py class">
@@ -1900,6 +1927,15 @@ the online documentation for more information.</p></li>
<span class="sig-name descname"><span class="pre">_udwf</span></span><a class="headerlink" href="#datafusion.user_defined.WindowUDF._udwf" title="Link to this definition">¶</a></dt>
<dd></dd></dl>
+<dl class="py property">
+<dt class="sig sig-object py" id="datafusion.user_defined.WindowUDF.name">
+<em class="property"><span class="pre">property</span><span class="w"> </span></em><span class="sig-name descname"><span class="pre">name</span></span><em class="property"><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="pre">str</span></em><a class="headerlink" href="#datafusion.user_defined.WindowUDF.name" title="Link to this definition">¶</a></dt>
+<dd><p>Return the registered name of this UDWF.</p>
+<p>For UDWFs imported via the FFI capsule protocol, this is the
+name the capsule itself reports — not the <code class="docutils literal notranslate"><span class="pre">name</span></code> argument
+passed to the constructor (which is ignored on the FFI path).</p>
+</dd></dl>
+
</dd></dl>
<dl class="py class">
diff --git a/genindex.html b/genindex.html
index c377140b..19e49599 100644
--- a/genindex.html
+++ b/genindex.html
@@ -2874,14 +2874,22 @@
<h2 id="N">N</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
- <li><a href="autoapi/datafusion/index.html#datafusion.Metric.name">name (datafusion.Metric property)</a>
+ <li><a href="autoapi/datafusion/index.html#datafusion.AggregateUDF.name">name (datafusion.AggregateUDF property)</a>
<ul>
+ <li><a href="autoapi/datafusion/index.html#datafusion.Metric.name">(datafusion.Metric property)</a>
+</li>
<li><a href="autoapi/datafusion/plan/index.html#datafusion.plan.Metric.name">(datafusion.plan.Metric property)</a>
</li>
<li><a href="autoapi/datafusion/index.html#datafusion.ScalarUDF.name">(datafusion.ScalarUDF property)</a>
+</li>
+ <li><a href="autoapi/datafusion/user_defined/index.html#datafusion.user_defined.AggregateUDF.name">(datafusion.user_defined.AggregateUDF property)</a>
</li>
<li><a href="autoapi/datafusion/user_defined/index.html#datafusion.user_defined.ScalarUDF.name">(datafusion.user_defined.ScalarUDF property)</a>
+</li>
+ <li><a href="autoapi/datafusion/user_defined/index.html#datafusion.user_defined.WindowUDF.name">(datafusion.user_defined.WindowUDF property)</a>
+</li>
+ <li><a href="autoapi/datafusion/index.html#datafusion.WindowUDF.name">(datafusion.WindowUDF property)</a>
</li>
</ul></li>
<li><a href="autoapi/datafusion/functions/index.html#datafusion.functions.named_struct">named_struct() (in module datafusion.functions)</a>
@@ -2900,14 +2908,14 @@
</li>
<li><a href="autoapi/datafusion/expr/index.html#datafusion.expr.Negative">Negative (in module datafusion.expr)</a>
</li>
+ </ul></td>
+ <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="autoapi/datafusion/index.html#datafusion.CsvReadOptions.newlines_in_values">newlines_in_values (datafusion.CsvReadOptions attribute)</a>
<ul>
<li><a href="autoapi/datafusion/options/index.html#datafusion.options.CsvReadOptions.newlines_in_values">(datafusion.options.CsvReadOptions attribute)</a>
</li>
</ul></li>
- </ul></td>
- <td style="width: 33%; vertical-align: top;"><ul>
<li><a href="autoapi/datafusion/record_batch/index.html#datafusion.record_batch.RecordBatchStream.next">next() (datafusion.record_batch.RecordBatchStream method)</a>
<ul>
diff --git a/objects.inv b/objects.inv
index c921dcb6..f8e49ac7 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/searchindex.js b/searchindex.js
index 1856bb24..e10acdf5 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"API Reference": [[19, null], [37, "api-reference"]], "Additional Resources": [[39, "additional-resources"]], "Aggregate Functions": [[25, "aggregate-functions"], [32, "aggregate-functions"], [34, "aggregate-functions"]], "Aggregated vs Per-Partition Metrics": [[37, "aggregated-vs-per-partition-metrics"]], "Aggregation": [[25, null]], "Alternative Approach": [[20, "alternative-approach"]], "Apache Iceberg": [[36, "apache-iceberg"]], "Arrays": [[27, "arrays" [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"API Reference": [[19, null], [37, "api-reference"]], "Additional Resources": [[39, "additional-resources"]], "Aggregate Functions": [[25, "aggregate-functions"], [32, "aggregate-functions"], [34, "aggregate-functions"]], "Aggregated vs Per-Partition Metrics": [[37, "aggregated-vs-per-partition-metrics"]], "Aggregation": [[25, null]], "Alternative Approach": [[20, "alternative-approach"]], "Apache Iceberg": [[36, "apache-iceberg"]], "Arrays": [[27, "arrays" [...]
\ No newline at end of file
diff --git a/user-guide/common-operations/aggregations.html b/user-guide/common-operations/aggregations.html
index 852736ac..c8214ee1 100644
--- a/user-guide/common-operations/aggregations.html
+++ b/user-guide/common-operations/aggregations.html
@@ -486,6 +486,11 @@
Aggregate Functions
</a>
</li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#user-defined-aggregate-functions">
+ User-Defined Aggregate Functions
+ </a>
+ </li>
</ul>
</nav>
@@ -657,15 +662,15 @@ Pokemon set. Since there will be many entries of <code class="docutils literal n
<span class="o">+----------+--------------------------------------------------+</span>
<span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Type</span> <span class="mi">2</span> <span class="n">List</span> <span class="o">|</span>
<span class="o">+----------+--------------------------------------------------+</span>
-<span class="o">|</span> <span class="n">Water</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Dark</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Rock</span> <span class="o">|</span> <span class="p">[</span><span class="n">Ground</span><span class="p">,</span> <span class="n">Water</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Water</span> <span class="o">|</span> <span class="p">[</span><span class="n">Psychic</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Dark</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">,</span> <span class="p">,</span> <span class="n">Poison</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Rock</span> <span class="o">|</span> <span class="p">[</span><span class="n">Water</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Ghost</span> <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Ice</span> <span class="o">|</span> <span class="p">[</span><span class="n">Psychic</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Dragon</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="p">[,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Fire</span> <span class="o">|</span> <span class="p">[,</span> <span class="n">Dragon</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Fairy</span><span class="p">,</span> <span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="p">[,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="p">[</span><span class="n">Fairy</span><span class="p">,</span> <span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="p">[,</span> <span class="n">Ground</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Fighting</span> <span class="o">|</span> <span class="p">[]</span> <span class="o">|</span>
<span class="o">+----------+--------------------------------------------------+</span>
<span class="n">Data</span> <span class="n">truncated</span><span class="o">.</span>
@@ -681,14 +686,14 @@ entries entirely removed. The second is we can use the <code class="docutils lit
<span class="o">+--------+------------------------------------------------+</span>
<span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Type</span> <span class="mi">2</span> <span class="n">List</span> <span class="o">|</span>
<span class="o">+--------+------------------------------------------------+</span>
-<span class="o">|</span> <span class="n">Water</span> <span class="o">|</span> <span class="p">[</span><span class="n">Fighting</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Dark</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Ice</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Rock</span> <span class="o">|</span> <span class="p">[</span><span class="n">Water</span><span class="p">,</span> <span class="n">Ground</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Water</span> <span class="o">|</span> <span class="p">[</span><span class="n">Dark</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Rock</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Water</span><span class="p">,</span> <span class="n">Ground</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Ghost</span> <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Ice</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Dragon</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Fire</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Dragon</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Fairy</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Fire</span> <span class="o">|</span> <span class="p">[</span><span class="n">Dragon</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="p">[</span><span class="n">Fairy</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Ground</span> <span class="o">|</span> <span class="p">[</span><span class="n">Rock</span><span class="p">]</span> <span class="o">|</span>
<span class="o">+--------+------------------------------------------------+</span>
@@ -700,15 +705,15 @@ entries entirely removed. The second is we can use the <code class="docutils lit
<span class="o">+----------+------------------------------------------------+</span>
<span class="o">|</span> <span class="n">Type</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">Type</span> <span class="mi">2</span> <span class="n">List</span> <span class="o">|</span>
<span class="o">+----------+------------------------------------------------+</span>
-<span class="o">|</span> <span class="n">Water</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Dark</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Rock</span> <span class="o">|</span> <span class="p">[</span><span class="n">Ground</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Water</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Water</span> <span class="o">|</span> <span class="p">[</span><span class="n">Dark</span><span class="p">,</span> <span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">,</span> <span class="n">Flying</span><span class="p">,</span> <span class="n">Ice</span><span class="p">,</span> <span class="n">Fighting</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Rock</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">,</span> <span class="n">Water</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Ghost</span> <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Ice</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Dragon</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="p">[</span><span class="n">Psychic</span><span class="p">,</span> <span class="n">Poison</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Fire</span> <span class="o">|</span> <span class="p">[</span><span class="n">Dragon</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Grass</span> <span class="o">|</span> <span class="p">[</span><span class="n">Poison</span><span class="p">,</span> <span class="n">Psychic</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Fire</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Dragon</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Normal</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Fairy</span><span class="p">]</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="p">[</span><span class="n">Ground</span><span class="p">,</span> <span class="n">Flying</span><span class="p">]</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">Poison</span> <span class="o">|</span> <span class="p">[</span><span class="n">Flying</span><span class="p">,</span> <span class="n">Ground</span><span class="p">]</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">Fighting</span> <span class="o">|</span> <span class="o">|</span>
<span class="o">+----------+------------------------------------------------+</span>
<span class="n">Data</span> <span class="n">truncated</span><span class="o">.</span>
@@ -1225,6 +1230,24 @@ Where <code class="docutils literal notranslate"><span class="pre">grouping(Type
- <a class="reference internal" href="../../autoapi/datafusion/expr/index.html#datafusion.expr.GroupingSet.grouping_sets" title="datafusion.expr.GroupingSet.grouping_sets"><code class="xref py py-meth docutils literal notranslate"><span class="pre">datafusion.expr.GroupingSet.grouping_sets()</span></code></a></p></li>
</ol>
</section>
+<section id="user-defined-aggregate-functions">
+<h2>User-Defined Aggregate Functions<a class="headerlink" href="#user-defined-aggregate-functions" title="Link to this heading">¶</a></h2>
+<p>You can ship custom aggregations to the engine by subclassing
+<a class="reference internal" href="../../autoapi/datafusion/user_defined/index.html#datafusion.user_defined.Accumulator" title="datafusion.user_defined.Accumulator"><code class="xref py py-class docutils literal notranslate"><span class="pre">Accumulator</span></code></a> and registering it via
+<a class="reference internal" href="../../autoapi/datafusion/index.html#datafusion.udaf" title="datafusion.udaf"><code class="xref py py-func docutils literal notranslate"><span class="pre">udaf()</span></code></a>. See <a class="reference internal" href="../../autoapi/datafusion/user_defined/index.html#module-datafusion.user_defined" title="datafusion.user_defined"><code class="xref py py-mod docutils literal notranslate"><span class="pre">datafusion.user_defined</span></code></a> for
+the accumulator interface and worked examples.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serialization</p>
+<p>Python aggregate UDFs travel inline inside pickled or
+<a class="reference internal" href="../../autoapi/datafusion/expr/index.html#datafusion.expr.Expr.to_bytes" title="datafusion.expr.Expr.to_bytes"><code class="xref py py-meth docutils literal notranslate"><span class="pre">to_bytes()</span></code></a>-serialized expressions —
+the accumulator class is captured by value via <code class="xref py py-mod docutils literal notranslate"><span class="pre">cloudpickle</span></code>,
+so worker processes do not need to pre-register the UDF. Any names
+the accumulator resolves via <code class="docutils literal notranslate"><span class="pre">import</span></code> are captured <strong>by reference</strong>
+and must be importable on the receiving worker. See
+<a class="reference internal" href="../../autoapi/datafusion/ipc/index.html#module-datafusion.ipc" title="datafusion.ipc"><code class="xref py py-mod docutils literal notranslate"><span class="pre">datafusion.ipc</span></code></a> for the full IPC model and security caveats.</p>
+</div>
+</section>
</section>
diff --git a/user-guide/common-operations/basic-info.html b/user-guide/common-operations/basic-info.html
index df064f7b..7f5a47c2 100644
--- a/user-guide/common-operations/basic-info.html
+++ b/user-guide/common-operations/basic-info.html
@@ -466,11 +466,11 @@
<span class="o">+-----+---------+--------+--------+</span>
<span class="o">|</span> <span class="n">nrs</span> <span class="o">|</span> <span class="n">names</span> <span class="o">|</span> <span class="n">random</span> <span class="o">|</span> <span class="n">groups</span> <span class="o">|</span>
<span class="o">+-----+---------+--------+--------+</span>
-<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">156</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mi">969</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="n">java</span> <span class="o">|</span> <span class="mi">349</span> <span class="o">|</span> <span class="n">B</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="n">haskell</span> <span class="o">|</span> <span class="mi">942</span> <span class="o">|</span> <span class="n">C</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="n">go</span> <span class="o">|</span> <span class="mi">749</span> <span class="o">|</span> <span class="n">B</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">909</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mi">16</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">3</span> <span class="o">|</span> <span class="n">java</span> <span class="o">|</span> <span class="mi">168</span> <span class="o">|</span> <span class="n">B</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">4</span> <span class="o">|</span> <span class="n">haskell</span> <span class="o">|</span> <span class="mi">652</span> <span class="o">|</span> <span class="n">C</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="n">go</span> <span class="o">|</span> <span class="mi">354</span> <span class="o">|</span> <span class="n">B</span> <span class="o">|</span>
<span class="o">+-----+---------+--------+--------+</span>
</pre></div>
</div>
@@ -481,8 +481,8 @@
<span class="o">+-----+--------+--------+--------+</span>
<span class="o">|</span> <span class="n">nrs</span> <span class="o">|</span> <span class="n">names</span> <span class="o">|</span> <span class="n">random</span> <span class="o">|</span> <span class="n">groups</span> <span class="o">|</span>
<span class="o">+-----+--------+--------+--------+</span>
-<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">156</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mi">969</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">1</span> <span class="o">|</span> <span class="n">python</span> <span class="o">|</span> <span class="mi">909</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mi">16</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
<span class="o">+-----+--------+--------+--------+</span>
</pre></div>
</div>
@@ -500,11 +500,11 @@ passing them to an Arrow table, and then converting them to a pandas DataFrame.<
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="n">In</span> <span class="p">[</span><span class="mi">8</span><span class="p">]:</span> <span class="n">df</span><span class="o">.</span><span class="n">to_pandas</span><span class="p">()</span>
<span class="n">Out</span><span class="p">[</span><span class="mi">8</span><span class="p">]:</span>
<span class="n">nrs</span> <span class="n">names</span> <span class="n">random</span> <span class="n">groups</span>
-<span class="mi">0</span> <span class="mi">1</span> <span class="n">python</span> <span class="mi">156</span> <span class="n">A</span>
-<span class="mi">1</span> <span class="mi">2</span> <span class="n">ruby</span> <span class="mi">969</span> <span class="n">A</span>
-<span class="mi">2</span> <span class="mi">3</span> <span class="n">java</span> <span class="mi">349</span> <span class="n">B</span>
-<span class="mi">3</span> <span class="mi">4</span> <span class="n">haskell</span> <span class="mi">942</span> <span class="n">C</span>
-<span class="mi">4</span> <span class="mi">5</span> <span class="n">go</span> <span class="mi">749</span> <span class="n">B</span>
+<span class="mi">0</span> <span class="mi">1</span> <span class="n">python</span> <span class="mi">909</span> <span class="n">A</span>
+<span class="mi">1</span> <span class="mi">2</span> <span class="n">ruby</span> <span class="mi">16</span> <span class="n">A</span>
+<span class="mi">2</span> <span class="mi">3</span> <span class="n">java</span> <span class="mi">168</span> <span class="n">B</span>
+<span class="mi">3</span> <span class="mi">4</span> <span class="n">haskell</span> <span class="mi">652</span> <span class="n">C</span>
+<span class="mi">4</span> <span class="mi">5</span> <span class="n">go</span> <span class="mi">354</span> <span class="n">B</span>
</pre></div>
</div>
<p><a class="reference internal" href="../../autoapi/datafusion/dataframe/index.html#datafusion.dataframe.DataFrame.describe" title="datafusion.dataframe.DataFrame.describe"><code class="xref py py-func docutils literal notranslate"><span class="pre">describe()</span></code></a> shows a quick statistic summary of your data:</p>
@@ -516,11 +516,11 @@ passing them to an Arrow table, and then converting them to a pandas DataFrame.<
<span class="o">+------------+--------------------+-------+-------------------+--------+</span>
<span class="o">|</span> <span class="n">count</span> <span class="o">|</span> <span class="mf">5.0</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span> <span class="mf">5.0</span> <span class="o">|</span> <span class="mi">5</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">null_count</span> <span class="o">|</span> <span class="mf">0.0</span> <span class="o">|</span> <span class="mi">0</span> <span class="o">|</span> <span class="mf">0.0</span> <span class="o">|</span> <span class="mi">0</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">mean</span> <span class="o">|</span> <span class="mf">3.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">633.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">std</span> <span class="o">|</span> <span class="mf">1.5811388300841898</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">364.0116756369224</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
-<span class="o">|</span> <span class="nb">min</span> <span class="o">|</span> <span class="mf">1.0</span> <span class="o">|</span> <span class="n">go</span> <span class="o">|</span> <span class="mf">156.0</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
-<span class="o">|</span> <span class="nb">max</span> <span class="o">|</span> <span class="mf">5.0</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mf">969.0</span> <span class="o">|</span> <span class="n">C</span> <span class="o">|</span>
-<span class="o">|</span> <span class="n">median</span> <span class="o">|</span> <span class="mf">3.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">749.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">mean</span> <span class="o">|</span> <span class="mf">3.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">419.8</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">std</span> <span class="o">|</span> <span class="mf">1.5811388300841898</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">361.9464048723236</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
+<span class="o">|</span> <span class="nb">min</span> <span class="o">|</span> <span class="mf">1.0</span> <span class="o">|</span> <span class="n">go</span> <span class="o">|</span> <span class="mf">16.0</span> <span class="o">|</span> <span class="n">A</span> <span class="o">|</span>
+<span class="o">|</span> <span class="nb">max</span> <span class="o">|</span> <span class="mf">5.0</span> <span class="o">|</span> <span class="n">ruby</span> <span class="o">|</span> <span class="mf">909.0</span> <span class="o">|</span> <span class="n">C</span> <span class="o">|</span>
+<span class="o">|</span> <span class="n">median</span> <span class="o">|</span> <span class="mf">3.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span> <span class="mf">354.0</span> <span class="o">|</span> <span class="n">null</span> <span class="o">|</span>
<span class="o">+------------+--------------------+-------+-------------------+--------+</span>
</pre></div>
</div>
diff --git a/user-guide/common-operations/functions.html b/user-guide/common-operations/functions.html
index db92507d..64e92c1e 100644
--- a/user-guide/common-operations/functions.html
+++ b/user-guide/common-operations/functions.html
@@ -576,16 +576,16 @@ In here we will cover some of the more popular use cases. If you want to view al
<span class="o">+-------------------------------+</span>
<span class="o">|</span> <span class="n">now</span><span class="p">()</span> <span class="o">|</span>
<span class="o">+-------------------------------+</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
-<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">19</span><span class="n">T17</span><span class="p">:</span><span class="mi">16</span><span class="p">:</span><span class="mf">35.337626138</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
+<span class="o">|</span> <span class="mi">2026</span><span class="o">-</span><span class="mi">05</span><span class="o">-</span><span class="mi">20</span><span class="n">T23</span><span class="p">:</span><span class="mi">34</span><span class="p">:</span><span class="mf">02.643568135</span> <span class="o">|</span>
<span class="o">+-------------------------------+</span>
<span class="n">Data</span> <span class="n">truncated</span><span class="o">.</span>
</pre></div>
diff --git a/user-guide/common-operations/index.html b/user-guide/common-operations/index.html
index 952e929d..0bdc4cf5 100644
--- a/user-guide/common-operations/index.html
+++ b/user-guide/common-operations/index.html
@@ -489,12 +489,14 @@
<li class="toctree-l2"><a class="reference internal" href="aggregations.html#setting-parameters">Setting Parameters</a></li>
<li class="toctree-l2"><a class="reference internal" href="aggregations.html#grouping-sets">Grouping Sets</a></li>
<li class="toctree-l2"><a class="reference internal" href="aggregations.html#aggregate-functions">Aggregate Functions</a></li>
+<li class="toctree-l2"><a class="reference internal" href="aggregations.html#user-defined-aggregate-functions">User-Defined Aggregate Functions</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="windows.html">Window Functions</a><ul>
<li class="toctree-l2"><a class="reference internal" href="windows.html#setting-parameters">Setting Parameters</a></li>
<li class="toctree-l2"><a class="reference internal" href="windows.html#aggregate-functions">Aggregate Functions</a></li>
<li class="toctree-l2"><a class="reference internal" href="windows.html#available-functions">Available Functions</a></li>
+<li class="toctree-l2"><a class="reference internal" href="windows.html#user-defined-window-functions">User-Defined Window Functions</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="udf-and-udfa.html">User-Defined Functions</a><ul>
diff --git a/user-guide/common-operations/udf-and-udfa.html b/user-guide/common-operations/udf-and-udfa.html
index 445bc4e2..ec5b50e8 100644
--- a/user-guide/common-operations/udf-and-udfa.html
+++ b/user-guide/common-operations/udf-and-udfa.html
@@ -649,7 +649,7 @@ over column references and literals, so the optimizer can analyze it:</p>
<span class="n">In</span> <span class="p">[</span><span class="mi">29</span><span class="p">]:</span> <span class="nb">print</span><span class="p">(</span><span class="n">native_filtered</span><span class="o">.</span><span class="n">execution_plan</span><span class="p">()</span><span class="o">.</span><span class="n">display_indent</span><span class="p">())</span>
<span class="n">FilterExec</span><span class="p">:</span> <span class="n">brand</span><span class="o">@</span><span class="mi">1</span> <span class="o">=</span> <span class="n">A</span> <span class="n">AND</span> <span class="n">qty</span><span class="o">@</span><span class="mi">2</span> <span class="o">>=</span> <span class="mi">150</span>
<span class="n">RepartitionExec</span><span class="p">:</span> <span class="n">partitioning</span><span class="o">=</span><span class="n">RoundRobinBatch</span><span class="p">(</span><span class="mi">4</span><span class="p">),</span> <span class="n">input_partitions</span><span class="o">=</span><span class="mi">1</span>
- <span class="n">DataSourceExec</span><span class="p">:</span> <span class="n">file_groups</span><span class="o">=</span><span class="p">{</span><span class="mi">1</span> <span class="n">group</span><span class="p">:</span> <span class="p">[[</span><span class="n">tmp</span><span class="o">/</span><span class="n">tmp8j2hu6av</span><span class="o">/</span><span class="n">items</span><span class="o">.</span><span class="n">parquet</span><span class="p">]]},</span> <span class="n">projec [...]
+ <span class="n">DataSourceExec</span><span class="p">:</span> <span class="n">file_groups</span><span class="o">=</span><span class="p">{</span><span class="mi">1</span> <span class="n">group</span><span class="p">:</span> <span class="p">[[</span><span class="n">tmp</span><span class="o">/</span><span class="n">tmplop1hvv7</span><span class="o">/</span><span class="n">items</span><span class="o">.</span><span class="n">parquet</span><span class="p">]]},</span> <span class="n">projec [...]
</pre></div>
</div>
<p>Notice the <code class="docutils literal notranslate"><span class="pre">DataSourceExec</span></code> line. It carries three annotations the
@@ -682,7 +682,7 @@ bloom filter or dictionary is available to skip pages.</p></li>
<span class="n">In</span> <span class="p">[</span><span class="mi">33</span><span class="p">]:</span> <span class="nb">print</span><span class="p">(</span><span class="n">udf_filtered</span><span class="o">.</span><span class="n">execution_plan</span><span class="p">()</span><span class="o">.</span><span class="n">display_indent</span><span class="p">())</span>
<span class="n">FilterExec</span><span class="p">:</span> <span class="n">brand_qty_filter</span><span class="p">(</span><span class="n">CAST</span><span class="p">(</span><span class="n">brand</span><span class="o">@</span><span class="mi">1</span> <span class="n">AS</span> <span class="n">Utf8</span><span class="p">),</span> <span class="n">qty</span><span class="o">@</span><span class="mi">2</span><span class="p">)</span>
<span class="n">RepartitionExec</span><span class="p">:</span> <span class="n">partitioning</span><span class="o">=</span><span class="n">RoundRobinBatch</span><span class="p">(</span><span class="mi">4</span><span class="p">),</span> <span class="n">input_partitions</span><span class="o">=</span><span class="mi">1</span>
- <span class="n">DataSourceExec</span><span class="p">:</span> <span class="n">file_groups</span><span class="o">=</span><span class="p">{</span><span class="mi">1</span> <span class="n">group</span><span class="p">:</span> <span class="p">[[</span><span class="n">tmp</span><span class="o">/</span><span class="n">tmp8j2hu6av</span><span class="o">/</span><span class="n">items</span><span class="o">.</span><span class="n">parquet</span><span class="p">]]},</span> <span class="n">projec [...]
+ <span class="n">DataSourceExec</span><span class="p">:</span> <span class="n">file_groups</span><span class="o">=</span><span class="p">{</span><span class="mi">1</span> <span class="n">group</span><span class="p">:</span> <span class="p">[[</span><span class="n">tmp</span><span class="o">/</span><span class="n">tmplop1hvv7</span><span class="o">/</span><span class="n">items</span><span class="o">.</span><span class="n">parquet</span><span class="p">]]},</span> <span class="n">projec [...]
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">DataSourceExec</span></code> now carries only <code class="docutils literal notranslate"><span class="pre">predicate=brand_qty_filter(...)</span></code>.
diff --git a/user-guide/common-operations/windows.html b/user-guide/common-operations/windows.html
index 9cc5ecd0..8a2da44b 100644
--- a/user-guide/common-operations/windows.html
+++ b/user-guide/common-operations/windows.html
@@ -464,6 +464,11 @@
Available Functions
</a>
</li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#user-defined-window-functions">
+ User-Defined Window Functions
+ </a>
+ </li>
</ul>
</nav>
@@ -784,6 +789,24 @@ power in its <code class="docutils literal notranslate"><span class="pre">"
</li>
</ol>
</section>
+<section id="user-defined-window-functions">
+<h2>User-Defined Window Functions<a class="headerlink" href="#user-defined-window-functions" title="Link to this heading">¶</a></h2>
+<p>You can ship custom window functions to the engine by subclassing
+<a class="reference internal" href="../../autoapi/datafusion/user_defined/index.html#datafusion.user_defined.WindowEvaluator" title="datafusion.user_defined.WindowEvaluator"><code class="xref py py-class docutils literal notranslate"><span class="pre">WindowEvaluator</span></code></a> and registering it
+via <a class="reference internal" href="../../autoapi/datafusion/index.html#datafusion.udwf" title="datafusion.udwf"><code class="xref py py-func docutils literal notranslate"><span class="pre">udwf()</span></code></a>. See <a class="reference internal" href="../../autoapi/datafusion/user_defined/index.html#module-datafusion.user_defined" title="datafusion.user_defined"><code class="xref py py-mod docutils literal notranslate"><span class="pre">datafusion.user_defined</span></code></a>
+for the evaluator interface and worked examples.</p>
+<div class="admonition note">
+<p class="admonition-title">Note</p>
+<p>Serialization</p>
+<p>Python window UDFs travel inline inside pickled or
+<a class="reference internal" href="../../autoapi/datafusion/expr/index.html#datafusion.expr.Expr.to_bytes" title="datafusion.expr.Expr.to_bytes"><code class="xref py py-meth docutils literal notranslate"><span class="pre">to_bytes()</span></code></a>-serialized expressions —
+the evaluator class is captured by value via <code class="xref py py-mod docutils literal notranslate"><span class="pre">cloudpickle</span></code>, so
+worker processes do not need to pre-register the UDF. Any names the
+evaluator resolves via <code class="docutils literal notranslate"><span class="pre">import</span></code> are captured <strong>by reference</strong> and
+must be importable on the receiving worker. See
+<a class="reference internal" href="../../autoapi/datafusion/ipc/index.html#module-datafusion.ipc" title="datafusion.ipc"><code class="xref py py-mod docutils literal notranslate"><span class="pre">datafusion.ipc</span></code></a> for the full IPC model and security caveats.</p>
+</div>
+</section>
</section>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org