You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/08/20 04:03:39 UTC
(datafusion) branch asf-site updated: Publish built docs triggered by f1f0449a5327a03cc99817fcd0ac8531f0c015ec
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 2f4f72acd5 Publish built docs triggered by f1f0449a5327a03cc99817fcd0ac8531f0c015ec
2f4f72acd5 is described below
commit 2f4f72acd5bff8898cdb203b7d619f370878c31c
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Thu Aug 20 04:03:33 2026 +0000
Publish built docs triggered by f1f0449a5327a03cc99817fcd0ac8531f0c015ec
---
.../library-user-guide/upgrading/55.0.0.md.txt | 44 ++++++++++++++++++++++
_sources/user-guide/sql/window_functions.md.txt | 2 +
library-user-guide/upgrading/55.0.0.html | 41 ++++++++++++++++++++
searchindex.js | 2 +-
user-guide/sql/window_functions.html | 1 +
5 files changed, 89 insertions(+), 1 deletion(-)
diff --git a/_sources/library-user-guide/upgrading/55.0.0.md.txt b/_sources/library-user-guide/upgrading/55.0.0.md.txt
index ef2785edb7..c87dbc3b8a 100644
--- a/_sources/library-user-guide/upgrading/55.0.0.md.txt
+++ b/_sources/library-user-guide/upgrading/55.0.0.md.txt
@@ -1408,3 +1408,47 @@ let table_opts = TableParquetOptions::try_from(&proto_table_opts)?;
```
See [issue #24019](https://github.com/apache/datafusion/issues/24019) for details.
+
+### RANGE window frames with offsets now reject ORDER BY types without arithmetic
+
+A `RANGE` frame offset such as `1 PRECEDING` is measured in `ORDER BY` values
+rather than in rows, so the frame bound is computed as `current_value - 1`. That
+requires arithmetic on the `ORDER BY` type. For types that have no such
+arithmetic -- `Utf8`, `Binary`, `Boolean`, `List` and `Null` -- such frames are
+now rejected when the query is planned:
+
+```sql
+SELECT x, COUNT(*) OVER (ORDER BY x RANGE BETWEEN 1 PRECEDING AND CURRENT ROW)
+FROM (VALUES ('a'), ('b'), ('c')) t(x)
+ORDER BY x;
+-- Error during planning: RANGE with offset PRECEDING/FOLLOWING is not supported for ORDER BY type Utf8
+```
+
+What this replaces depends on the `ORDER BY` type. Over `Utf8`, `Boolean` and
+`List` keys such queries planned successfully, and in DataFusion 54 the failing
+frame-bound arithmetic was silently treated as an overflow
+([#22140](https://github.com/apache/datafusion/pull/22140)): the bound collapsed
+to the partition edge, and the query returned results computed over that wrongly
+widened frame instead of an offset window. DataFusion 53 and earlier failed
+during execution with an arithmetic error. Rows whose order key value was `NULL`
+never reached the arithmetic, so queries where every order key value was `NULL`
+returned correct results throughout; those also now fail at planning. Over a
+`Null`-typed key the offset itself failed to coerce, erroring at planning time
+with `Cast error: Casting from Utf8 to Null not supported`, and binary keys were
+rejected for every `RANGE` frame with the internal error quoted below -- for
+those two, only the error message changes.
+
+Such an `ORDER BY` type is still usable with the `UNBOUNDED PRECEDING`,
+`CURRENT ROW` and `UNBOUNDED FOLLOWING` bounds, which are located by comparing
+`ORDER BY` values rather than by computing them. Binary order keys additionally
+gained support for those bounds in this release, having previously failed with
+`Internal error: Cannot run range queries on datatype: Binary`.
+
+**Who is affected:**
+
+- Queries using `offset PRECEDING` or `offset FOLLOWING` in a `RANGE` frame over
+ a non-numeric, non-temporal `ORDER BY` key. To count rows rather than compare
+ values, use a `ROWS` frame; to include everything up to the partition edge,
+ use `UNBOUNDED PRECEDING` or `UNBOUNDED FOLLOWING`.
+
+See [issue #24327](https://github.com/apache/datafusion/issues/24327) for details.
diff --git a/_sources/user-guide/sql/window_functions.md.txt b/_sources/user-guide/sql/window_functions.md.txt
index 2c8050ce1f..6496da1102 100644
--- a/_sources/user-guide/sql/window_functions.md.txt
+++ b/_sources/user-guide/sql/window_functions.md.txt
@@ -145,6 +145,8 @@ where **offset** is an non-negative integer.
RANGE and GROUPS modes require an ORDER BY clause (with RANGE the ORDER BY must specify exactly one column).
+In RANGE mode an **offset** is measured in ORDER BY values rather than in rows, so the bound is computed by adding it to or subtracting it from the current row's ORDER BY value. That restricts `offset PRECEDING` and `offset FOLLOWING` to ORDER BY types supporting such arithmetic, namely the numeric, date, and timestamp types. Other orderable types, such as strings and binaries, can still be used with `UNBOUNDED PRECEDING`, `CURRENT ROW` and `UNBOUNDED FOLLOWING`, which are located by com [...]
+
## Filter clause for aggregate window functions
Aggregate window functions support the SQL `FILTER (WHERE ...)` clause to include only rows that satisfy the predicate from the window frame in the aggregation.
diff --git a/library-user-guide/upgrading/55.0.0.html b/library-user-guide/upgrading/55.0.0.html
index 2439dfc650..9285bb0db1 100644
--- a/library-user-guide/upgrading/55.0.0.html
+++ b/library-user-guide/upgrading/55.0.0.html
@@ -1663,6 +1663,46 @@ changes.</p>
</div>
<p>See <a class="reference external" href="https://github.com/apache/datafusion/issues/24019">issue #24019</a> for details.</p>
</section>
+<section id="range-window-frames-with-offsets-now-reject-order-by-types-without-arithmetic">
+<h3>RANGE window frames with offsets now reject ORDER BY types without arithmetic<a class="headerlink" href="#range-window-frames-with-offsets-now-reject-order-by-types-without-arithmetic" title="Link to this heading">#</a></h3>
+<p>A <code class="docutils literal notranslate"><span class="pre">RANGE</span></code> frame offset such as <code class="docutils literal notranslate"><span class="pre">1</span> <span class="pre">PRECEDING</span></code> is measured in <code class="docutils literal notranslate"><span class="pre">ORDER</span> <span class="pre">BY</span></code> values
+rather than in rows, so the frame bound is computed as <code class="docutils literal notranslate"><span class="pre">current_value</span> <span class="pre">-</span> <span class="pre">1</span></code>. That
+requires arithmetic on the <code class="docutils literal notranslate"><span class="pre">ORDER</span> <span class="pre">BY</span></code> type. For types that have no such
+arithmetic – <code class="docutils literal notranslate"><span class="pre">Utf8</span></code>, <code class="docutils literal notranslate"><span class="pre">Binary</span></code>, <code class="docutils literal notranslate"><span class="pre">Boolean</span></code>, <code class="docutils literal notranslate"><span class="pre">List</span></code> and <code class="docutils literal notranslate"><span class="pre">Null</span></code> – such frames are
+now rejected when the query is planned:</p>
+<div class="highlight-sql notranslate"><div class="highlight"><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="n">x</span><span class="p">,</span><span class="w"> </span><span class="k">COUNT</span><span class="p">(</span><span class="o">*</span><span class="p">)</span><span class="w"> </span><span class="n">OVER</span><span class="w"> </span><span class="p">(</span><span class="k">ORDER</span><span class="w"> </span><span class="k">BY</span><span class [...]
+<span class="k">FROM</span><span class="w"> </span><span class="p">(</span><span class="k">VALUES</span><span class="w"> </span><span class="p">(</span><span class="s1">'a'</span><span class="p">),</span><span class="w"> </span><span class="p">(</span><span class="s1">'b'</span><span class="p">),</span><span class="w"> </span><span class="p">(</span><span class="s1">'c'</span><span class="p">))</span><span class="w"> </span><span class="n">t</span><span class="p"> [...]
+<span class="k">ORDER</span><span class="w"> </span><span class="k">BY</span><span class="w"> </span><span class="n">x</span><span class="p">;</span>
+<span class="c1">-- Error during planning: RANGE with offset PRECEDING/FOLLOWING is not supported for ORDER BY type Utf8</span>
+</pre></div>
+</div>
+<p>What this replaces depends on the <code class="docutils literal notranslate"><span class="pre">ORDER</span> <span class="pre">BY</span></code> type. Over <code class="docutils literal notranslate"><span class="pre">Utf8</span></code>, <code class="docutils literal notranslate"><span class="pre">Boolean</span></code> and
+<code class="docutils literal notranslate"><span class="pre">List</span></code> keys such queries planned successfully, and in DataFusion 54 the failing
+frame-bound arithmetic was silently treated as an overflow
+(<a class="reference external" href="https://github.com/apache/datafusion/pull/22140">#22140</a>): the bound collapsed
+to the partition edge, and the query returned results computed over that wrongly
+widened frame instead of an offset window. DataFusion 53 and earlier failed
+during execution with an arithmetic error. Rows whose order key value was <code class="docutils literal notranslate"><span class="pre">NULL</span></code>
+never reached the arithmetic, so queries where every order key value was <code class="docutils literal notranslate"><span class="pre">NULL</span></code>
+returned correct results throughout; those also now fail at planning. Over a
+<code class="docutils literal notranslate"><span class="pre">Null</span></code>-typed key the offset itself failed to coerce, erroring at planning time
+with <code class="docutils literal notranslate"><span class="pre">Cast</span> <span class="pre">error:</span> <span class="pre">Casting</span> <span class="pre">from</span> <span class="pre">Utf8</span> <span class="pre">to</span> <span class="pre">Null</span> <span class="pre">not</span> <span class="pre">supported</span></code>, and binary keys were
+rejected for every <code class="docutils literal notranslate"><span class="pre">RANGE</span></code> frame with the internal error quoted below – for
+those two, only the error message changes.</p>
+<p>Such an <code class="docutils literal notranslate"><span class="pre">ORDER</span> <span class="pre">BY</span></code> type is still usable with the <code class="docutils literal notranslate"><span class="pre">UNBOUNDED</span> <span class="pre">PRECEDING</span></code>,
+<code class="docutils literal notranslate"><span class="pre">CURRENT</span> <span class="pre">ROW</span></code> and <code class="docutils literal notranslate"><span class="pre">UNBOUNDED</span> <span class="pre">FOLLOWING</span></code> bounds, which are located by comparing
+<code class="docutils literal notranslate"><span class="pre">ORDER</span> <span class="pre">BY</span></code> values rather than by computing them. Binary order keys additionally
+gained support for those bounds in this release, having previously failed with
+<code class="docutils literal notranslate"><span class="pre">Internal</span> <span class="pre">error:</span> <span class="pre">Cannot</span> <span class="pre">run</span> <span class="pre">range</span> <span class="pre">queries</span> <span class="pre">on</span> <span class="pre">datatype:</span> <span class="pre">Binary</span></code>.</p>
+<p><strong>Who is affected:</strong></p>
+<ul class="simple">
+<li><p>Queries using <code class="docutils literal notranslate"><span class="pre">offset</span> <span class="pre">PRECEDING</span></code> or <code class="docutils literal notranslate"><span class="pre">offset</span> <span class="pre">FOLLOWING</span></code> in a <code class="docutils literal notranslate"><span class="pre">RANGE</span></code> frame over
+a non-numeric, non-temporal <code class="docutils literal notranslate"><span class="pre">ORDER</span> <span class="pre">BY</span></code> key. To count rows rather than compare
+values, use a <code class="docutils literal notranslate"><span class="pre">ROWS</span></code> frame; to include everything up to the partition edge,
+use <code class="docutils literal notranslate"><span class="pre">UNBOUNDED</span> <span class="pre">PRECEDING</span></code> or <code class="docutils literal notranslate"><span class="pre">UNBOUNDED</span> <span class="pre">FOLLOWING</span></code>.</p></li>
+</ul>
+<p>See <a class="reference external" href="https://github.com/apache/datafusion/issues/24327">issue #24327</a> for details.</p>
+</section>
</section>
</section>
@@ -1753,6 +1793,7 @@ changes.</p>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#exprtype-gained-a-sqlsimilartopattern-variant"><code class="docutils literal notranslate"><span class="pre">ExprType</span></code> gained a <code class="docutils literal notranslate"><span class="pre">SqlSimilarToPattern</span></code> variant</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#parquetobjectreader-parquetobjectwriter-deprecated-upstream"><code class="docutils literal notranslate"><span class="pre">ParquetObjectReader</span></code> / <code class="docutils literal notranslate"><span class="pre">ParquetObjectWriter</span></code> deprecated upstream</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#datafusion-proto-parquet-options-conversions-are-fallible"><code class="docutils literal notranslate"><span class="pre">datafusion-proto</span></code>: parquet options conversions are fallible</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#range-window-frames-with-offsets-now-reject-order-by-types-without-arithmetic">RANGE window frames with offsets now reject ORDER BY types without arithmetic</a></li>
</ul>
</li>
</ul>
diff --git a/searchindex.js b/searchindex.js
index cce87d7ea2..5f23965423 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[76,"op-neq"]],"!~":[[76,"op-re-not-match"]],"!~*":[[76,"op-re-not-match-i"]],"!~~":[[76,"id19"]],"!~~*":[[76,"id20"]],"#":[[76,"op-bit-xor"]],"%":[[76,"op-modulo"]],"&":[[76,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[16,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[76,"op-multiply"]],"+":[[76,"op-plus"]],"-":[[76,"op-minus"]],"/":[[76,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[76,"op-neq"]],"!~":[[76,"op-re-not-match"]],"!~*":[[76,"op-re-not-match-i"]],"!~~":[[76,"id19"]],"!~~*":[[76,"id20"]],"#":[[76,"op-bit-xor"]],"%":[[76,"op-modulo"]],"&":[[76,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[16,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[76,"op-multiply"]],"+":[[76,"op-plus"]],"-":[[76,"op-minus"]],"/":[[76,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
diff --git a/user-guide/sql/window_functions.html b/user-guide/sql/window_functions.html
index 272139d0f0..550f2f1b82 100644
--- a/user-guide/sql/window_functions.html
+++ b/user-guide/sql/window_functions.html
@@ -557,6 +557,7 @@ Instead, the rows retain their separate identities. Behind the scenes, the windo
</div>
<p>where <strong>offset</strong> is an non-negative integer.</p>
<p>RANGE and GROUPS modes require an ORDER BY clause (with RANGE the ORDER BY must specify exactly one column).</p>
+<p>In RANGE mode an <strong>offset</strong> is measured in ORDER BY values rather than in rows, so the bound is computed by adding it to or subtracting it from the current row’s ORDER BY value. That restricts <code class="docutils literal notranslate"><span class="pre">offset</span> <span class="pre">PRECEDING</span></code> and <code class="docutils literal notranslate"><span class="pre">offset</span> <span class="pre">FOLLOWING</span></code> to ORDER BY types supporting such arithmetic, [...]
</section>
<section id="filter-clause-for-aggregate-window-functions">
<h2>Filter clause for aggregate window functions<a class="headerlink" href="#filter-clause-for-aggregate-window-functions" title="Link to this heading">#</a></h2>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org