You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/05/28 23:15:33 UTC

(datafusion-comet) branch asf-site updated (3fd74525ce -> d94708ab21)

This is an automated email from the ASF dual-hosted git repository.

github-actions[bot] pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git


 discard 3fd74525ce Publish built docs triggered by 598a32d0dd1bc7340079640578962c6e4a5cc933
     new d94708ab21 Publish built docs triggered by bedd6b706ee5978f077a63f62ef948a65d70afcc

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (3fd74525ce)
            \
             N -- N -- N   refs/heads/asf-site (d94708ab21)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../spark_expressions_support.md.txt               | 15 +++++++++++
 .../latest/compatibility/expressions/string.md.txt |  2 +-
 contributor-guide/spark_expressions_support.html   | 30 +++++++++++++++++++---
 searchindex.js                                     |  2 +-
 .../latest/compatibility/expressions/string.html   |  2 +-
 5 files changed, 45 insertions(+), 6 deletions(-)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org


(datafusion-comet) 01/01: Publish built docs triggered by bedd6b706ee5978f077a63f62ef948a65d70afcc

Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git

commit d94708ab213f19e107d866c4f9ae87d5693bd90b
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Thu May 28 23:15:28 2026 +0000

    Publish built docs triggered by bedd6b706ee5978f077a63f62ef948a65d70afcc
---
 .../spark_expressions_support.md.txt               | 37 +++++++++++
 .../latest/compatibility/expressions/math.md.txt   | 12 ++++
 .../latest/compatibility/expressions/string.md.txt |  2 +-
 contributor-guide/spark_expressions_support.html   | 77 +++++++++++++++++++---
 searchindex.js                                     |  2 +-
 .../latest/compatibility/expressions/index.html    |  2 +
 .../latest/compatibility/expressions/math.html     | 14 ++++
 .../latest/compatibility/expressions/string.html   |  2 +-
 8 files changed, 137 insertions(+), 11 deletions(-)

diff --git a/_sources/contributor-guide/spark_expressions_support.md.txt b/_sources/contributor-guide/spark_expressions_support.md.txt
index 9cc37ccc29..6b3d122708 100644
--- a/_sources/contributor-guide/spark_expressions_support.md.txt
+++ b/_sources/contributor-guide/spark_expressions_support.md.txt
@@ -311,8 +311,23 @@
 - [ ] array_size
 - [ ] cardinality
 - [x] concat
+  - Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.
+  - Spark 3.5.8 (audited 2026-05-27): baseline. `Concat(children) extends ComplexTypeMergingExpression with QueryErrorsBase`; `allowedTypes = Seq(StringType, BinaryType, ArrayType)`; result type is the merged child type. Empty children is allowed and returns the empty string of the result type.
+  - Spark 4.0.1 (audited 2026-05-27): `allowedTypes` widens `StringType` to `StringTypeWithCollation(supportsTrimCollation = true)`. Error-formatting helper changes from `paramIndex` to `ordinalNumber`. Runtime semantics unchanged for `UTF8_BINARY`.
+  - Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.
+  - Known limitation: Comet only supports `StringType` children natively; `BinaryType` and `ArrayType` inputs fall back to Spark (https://github.com/apache/datafusion-comet/issues/4471). Non-default Spark 4.0 string collations are not propagated (https://github.com/apache/datafusion-comet/issues/2190).
 - [x] reverse
+  - Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.
+  - Spark 3.5.8 (audited 2026-05-27): baseline. `Reverse(child) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant`; `inputTypes = Seq(TypeCollection(StringType, ArrayType))`; `dataType = child.dataType`. For string, calls `UTF8String.reverse()`; for array, reverses element order in-place via `GenericArrayData`.
+  - Spark 4.0.1 (audited 2026-05-27): `NullIntolerant` trait replaced by `override def nullIntolerant: Boolean = true`; `inputTypes` widened to `Seq(TypeCollection(StringTypeWithCollation(supportsTrimCollation = true), ArrayType))`. Semantics unchanged for `UTF8_BINARY`.
+  - Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.
+  - Known limitation: `Reverse` on an array containing `BinaryType` elements is reported as `Incompatible` and falls back unless explicitly enabled (https://github.com/apache/datafusion-comet/issues/2763).
 - [x] size
+  - Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.
+  - Spark 3.5.8 (audited 2026-05-27): baseline. `Size(child, legacySizeOfNull) extends UnaryExpression with ExpectsInputTypes`; `inputTypes = Seq(TypeCollection(ArrayType, MapType)) -> IntegerType`. `legacySizeOfNull=true` returns `-1` for NULL input; `false` returns NULL. Comet routes via `CometSize`, which emits a `CaseWhen(isNotNull(child), size_scalar(child), Literal(legacySizeOfNull))`.
+  - Spark 4.0.1 (audited 2026-05-27): byte-for-byte identical to 3.5.8.
+  - Spark 4.1.1 (audited 2026-05-27): byte-for-byte identical to 3.5.8.
+  - Known limitation: `Size` over `MapType` falls back to Spark (https://github.com/apache/datafusion-comet/issues/4472).
 
 ### conditional_funcs
 
@@ -730,16 +745,34 @@
 - [ ] is_variant_null
 - [ ] java_method
 - [x] monotonically_increasing_id
+  - Spark 3.4.3 (audited 2026-05-27): byte-for-byte identical to 4.1.1. `MonotonicallyIncreasingID() extends LeafExpression with Stateful`; produces a Long that encodes the partition id in the upper 31 bits and a per-partition row counter in the lower 33 bits. Comet emits an empty `MonotonicallyIncreasingId` proto and the native side produces the same encoding.
+  - Spark 3.5.8 (audited 2026-05-27): identical to 3.4.3.
+  - Spark 4.0.1 (audited 2026-05-27): identical to 3.4.3.
+  - Spark 4.1.1 (audited 2026-05-27): identical to 3.4.3.
 - [ ] parse_json
 - [ ] raise_error
 - [x] rand
+  - Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.
+  - Spark 3.5.8 (audited 2026-05-27): baseline. `Rand(child, hideSeed) extends RDG` (an `UnaryExpression with ExpectsInputTypes with Nondeterministic with ExpressionWithRandomSeed`); `child` is the seed expression, coerced to `IntegerType` or `LongType` via `ImplicitCastInputTypes`. Uses `XORShiftRandom(seed + partitionIndex)` per partition and returns `nextDouble()` in `[0, 1)`. NULL seed evaluates to `0L` (via `null.asInstanceOf[Long]`).
+  - Spark 4.0.1 (audited 2026-05-27): `RDG` is refactored from an `abstract class` into a trait, and `Rand` now extends a new `NondeterministicUnaryRDG` base. `ExpressionWithRandomSeed.expressionToSeed` is hoisted as a shared helper and throws `QueryCompilationErrors.invalidRandomSeedParameter` for non-literal seeds at analysis time. Runtime semantics unchanged.
+  - Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.
+  - Comet limitation: the seed argument must be a literal (column-reference seeds are rejected via `getSupportLevel`). Pre-4.0 Spark would otherwise silently fail at runtime; 4.0+ rejects at analysis time before the expression reaches Comet.
 - [x] randn
+  - Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.
+  - Spark 3.5.8 (audited 2026-05-27): same base as `Rand`; differs only in the eval body (`nextGaussian()` instead of `nextDouble()`), producing values from the standard normal distribution.
+  - Spark 4.0.1 (audited 2026-05-27): same refactor as `Rand`; runtime unchanged.
+  - Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.
+  - Comet limitation: same as `rand` — the seed argument must be a literal.
 - [ ] reflect
 - [ ] schema_of_avro
 - [ ] schema_of_variant
 - [ ] schema_of_variant_agg
 - [ ] session_user
 - [x] spark_partition_id
+  - Spark 3.4.3 (audited 2026-05-27): byte-for-byte identical to 4.1.1. `SparkPartitionID() extends LeafExpression with Nondeterministic`; returns the integer index of the partition being processed. Comet emits an empty `SparkPartitionId` proto.
+  - Spark 3.5.8 (audited 2026-05-27): identical to 3.4.3.
+  - Spark 4.0.1 (audited 2026-05-27): identical to 3.4.3.
+  - Spark 4.1.1 (audited 2026-05-27): identical to 3.4.3.
 - [ ] st_asbinary
 - [ ] st_geogfromwkb
 - [ ] st_geomfromwkb
@@ -758,6 +791,10 @@
 - [ ] try_variant_get
 - [ ] typeof
 - [x] user
+  - Spark 3.4.3 (audited 2026-05-27): `CurrentUser() extends LeafExpression with Unevaluable`; the analyzer's `ResolveCurrentLike` rule replaces it with a `StringType` literal of the current user name before Comet sees the plan. No Comet serde needed; the literal flows through `CometLiteral`.
+  - Spark 3.5.8 (audited 2026-05-27): identical to 3.4.3.
+  - Spark 4.0.1 (audited 2026-05-27): identical to 3.4.3 except the resulting literal carries the default string collation.
+  - Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.
 - [ ] uuid
 - [ ] variant_get
 - [ ] version
diff --git a/_sources/user-guide/latest/compatibility/expressions/math.md.txt b/_sources/user-guide/latest/compatibility/expressions/math.md.txt
index f347f697e8..8b2aefc327 100644
--- a/_sources/user-guide/latest/compatibility/expressions/math.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/math.md.txt
@@ -26,4 +26,16 @@ under the License.
 The following cases are not supported by Comet:
 
 - Only integral, floating-point, and decimal types are supported
+
+## Rand
+
+The following cases are not supported by Comet:
+
+- The `seed` argument must be a literal value
+
+## Randn
+
+The following cases are not supported by Comet:
+
+- The `seed` argument must be a literal value
 <!--END:EXPR_COMPAT-->
diff --git a/_sources/user-guide/latest/compatibility/expressions/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/string.md.txt
index dc74667765..9e8d97a60c 100644
--- a/_sources/user-guide/latest/compatibility/expressions/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/string.md.txt
@@ -23,7 +23,7 @@ under the License.
 
 ## Concat
 
-The following incompatibilities cause `Concat` to fall back to Spark by default. Set `spark.comet.expression.Concat.allowIncompatible=true` to enable Comet acceleration despite these differences.
+The following cases are not supported by Comet:
 
 - CONCAT supports only string input parameters
 
diff --git a/contributor-guide/spark_expressions_support.html b/contributor-guide/spark_expressions_support.html
index fecf1f8c16..ae9bb32dac 100644
--- a/contributor-guide/spark_expressions_support.html
+++ b/contributor-guide/spark_expressions_support.html
@@ -929,9 +929,33 @@ per-function details are documented in the
 <ul class="contains-task-list simple">
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> array_size</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> cardinality</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> concat</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> reverse</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> size</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> concat</p>
+<ul>
+<li><p>Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.</p></li>
+<li><p>Spark 3.5.8 (audited 2026-05-27): baseline. <code class="docutils literal notranslate"><span class="pre">Concat(children)</span> <span class="pre">extends</span> <span class="pre">ComplexTypeMergingExpression</span> <span class="pre">with</span> <span class="pre">QueryErrorsBase</span></code>; <code class="docutils literal notranslate"><span class="pre">allowedTypes</span> <span class="pre">=</span> <span class="pre">Seq(StringType,</span> <span class="pre">BinaryType,</span> <spa [...]
+<li><p>Spark 4.0.1 (audited 2026-05-27): <code class="docutils literal notranslate"><span class="pre">allowedTypes</span></code> widens <code class="docutils literal notranslate"><span class="pre">StringType</span></code> to <code class="docutils literal notranslate"><span class="pre">StringTypeWithCollation(supportsTrimCollation</span> <span class="pre">=</span> <span class="pre">true)</span></code>. Error-formatting helper changes from <code class="docutils literal notranslate"><span c [...]
+<li><p>Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.</p></li>
+<li><p>Known limitation: Comet only supports <code class="docutils literal notranslate"><span class="pre">StringType</span></code> children natively; <code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> and <code class="docutils literal notranslate"><span class="pre">ArrayType</span></code> inputs fall back to Spark (https://github.com/apache/datafusion-comet/issues/4471). Non-default Spark 4.0 string collations are not propagated (https://github.com/apac [...]
+</ul>
+</li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> reverse</p>
+<ul>
+<li><p>Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.</p></li>
+<li><p>Spark 3.5.8 (audited 2026-05-27): baseline. <code class="docutils literal notranslate"><span class="pre">Reverse(child)</span> <span class="pre">extends</span> <span class="pre">UnaryExpression</span> <span class="pre">with</span> <span class="pre">ImplicitCastInputTypes</span> <span class="pre">with</span> <span class="pre">NullIntolerant</span></code>; <code class="docutils literal notranslate"><span class="pre">inputTypes</span> <span class="pre">=</span> <span class="pre">Seq( [...]
+<li><p>Spark 4.0.1 (audited 2026-05-27): <code class="docutils literal notranslate"><span class="pre">NullIntolerant</span></code> trait replaced by <code class="docutils literal notranslate"><span class="pre">override</span> <span class="pre">def</span> <span class="pre">nullIntolerant:</span> <span class="pre">Boolean</span> <span class="pre">=</span> <span class="pre">true</span></code>; <code class="docutils literal notranslate"><span class="pre">inputTypes</span></code> widened to < [...]
+<li><p>Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.</p></li>
+<li><p>Known limitation: <code class="docutils literal notranslate"><span class="pre">Reverse</span></code> on an array containing <code class="docutils literal notranslate"><span class="pre">BinaryType</span></code> elements is reported as <code class="docutils literal notranslate"><span class="pre">Incompatible</span></code> and falls back unless explicitly enabled (https://github.com/apache/datafusion-comet/issues/2763).</p></li>
+</ul>
+</li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> size</p>
+<ul>
+<li><p>Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.</p></li>
+<li><p>Spark 3.5.8 (audited 2026-05-27): baseline. <code class="docutils literal notranslate"><span class="pre">Size(child,</span> <span class="pre">legacySizeOfNull)</span> <span class="pre">extends</span> <span class="pre">UnaryExpression</span> <span class="pre">with</span> <span class="pre">ExpectsInputTypes</span></code>; <code class="docutils literal notranslate"><span class="pre">inputTypes</span> <span class="pre">=</span> <span class="pre">Seq(TypeCollection(ArrayType,</span> <s [...]
+<li><p>Spark 4.0.1 (audited 2026-05-27): byte-for-byte identical to 3.5.8.</p></li>
+<li><p>Spark 4.1.1 (audited 2026-05-27): byte-for-byte identical to 3.5.8.</p></li>
+<li><p>Known limitation: <code class="docutils literal notranslate"><span class="pre">Size</span></code> over <code class="docutils literal notranslate"><span class="pre">MapType</span></code> falls back to Spark (https://github.com/apache/datafusion-comet/issues/4472).</p></li>
+</ul>
+</li>
 </ul>
 </section>
 <section id="conditional-funcs">
@@ -1607,17 +1631,47 @@ per-function details are documented in the
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> input_file_name</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> is_variant_null</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> java_method</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> monotonically_increasing_id</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> monotonically_increasing_id</p>
+<ul>
+<li><p>Spark 3.4.3 (audited 2026-05-27): byte-for-byte identical to 4.1.1. <code class="docutils literal notranslate"><span class="pre">MonotonicallyIncreasingID()</span> <span class="pre">extends</span> <span class="pre">LeafExpression</span> <span class="pre">with</span> <span class="pre">Stateful</span></code>; produces a Long that encodes the partition id in the upper 31 bits and a per-partition row counter in the lower 33 bits. Comet emits an empty <code class="docutils literal notr [...]
+<li><p>Spark 3.5.8 (audited 2026-05-27): identical to 3.4.3.</p></li>
+<li><p>Spark 4.0.1 (audited 2026-05-27): identical to 3.4.3.</p></li>
+<li><p>Spark 4.1.1 (audited 2026-05-27): identical to 3.4.3.</p></li>
+</ul>
+</li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> parse_json</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> raise_error</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> rand</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> randn</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> rand</p>
+<ul>
+<li><p>Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.</p></li>
+<li><p>Spark 3.5.8 (audited 2026-05-27): baseline. <code class="docutils literal notranslate"><span class="pre">Rand(child,</span> <span class="pre">hideSeed)</span> <span class="pre">extends</span> <span class="pre">RDG</span></code> (an <code class="docutils literal notranslate"><span class="pre">UnaryExpression</span> <span class="pre">with</span> <span class="pre">ExpectsInputTypes</span> <span class="pre">with</span> <span class="pre">Nondeterministic</span> <span class="pre">with</ [...]
+<li><p>Spark 4.0.1 (audited 2026-05-27): <code class="docutils literal notranslate"><span class="pre">RDG</span></code> is refactored from an <code class="docutils literal notranslate"><span class="pre">abstract</span> <span class="pre">class</span></code> into a trait, and <code class="docutils literal notranslate"><span class="pre">Rand</span></code> now extends a new <code class="docutils literal notranslate"><span class="pre">NondeterministicUnaryRDG</span></code> base. <code class=" [...]
+<li><p>Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.</p></li>
+<li><p>Comet limitation: the seed argument must be a literal (column-reference seeds are rejected via <code class="docutils literal notranslate"><span class="pre">getSupportLevel</span></code>). Pre-4.0 Spark would otherwise silently fail at runtime; 4.0+ rejects at analysis time before the expression reaches Comet.</p></li>
+</ul>
+</li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> randn</p>
+<ul>
+<li><p>Spark 3.4.3 (audited 2026-05-27): identical to 3.5.8.</p></li>
+<li><p>Spark 3.5.8 (audited 2026-05-27): same base as <code class="docutils literal notranslate"><span class="pre">Rand</span></code>; differs only in the eval body (<code class="docutils literal notranslate"><span class="pre">nextGaussian()</span></code> instead of <code class="docutils literal notranslate"><span class="pre">nextDouble()</span></code>), producing values from the standard normal distribution.</p></li>
+<li><p>Spark 4.0.1 (audited 2026-05-27): same refactor as <code class="docutils literal notranslate"><span class="pre">Rand</span></code>; runtime unchanged.</p></li>
+<li><p>Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.</p></li>
+<li><p>Comet limitation: same as <code class="docutils literal notranslate"><span class="pre">rand</span></code> — the seed argument must be a literal.</p></li>
+</ul>
+</li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> reflect</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> schema_of_avro</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> schema_of_variant</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> schema_of_variant_agg</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> session_user</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> spark_partition_id</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> spark_partition_id</p>
+<ul>
+<li><p>Spark 3.4.3 (audited 2026-05-27): byte-for-byte identical to 4.1.1. <code class="docutils literal notranslate"><span class="pre">SparkPartitionID()</span> <span class="pre">extends</span> <span class="pre">LeafExpression</span> <span class="pre">with</span> <span class="pre">Nondeterministic</span></code>; returns the integer index of the partition being processed. Comet emits an empty <code class="docutils literal notranslate"><span class="pre">SparkPartitionId</span></code> prot [...]
+<li><p>Spark 3.5.8 (audited 2026-05-27): identical to 3.4.3.</p></li>
+<li><p>Spark 4.0.1 (audited 2026-05-27): identical to 3.4.3.</p></li>
+<li><p>Spark 4.1.1 (audited 2026-05-27): identical to 3.4.3.</p></li>
+</ul>
+</li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> st_asbinary</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> st_geogfromwkb</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> st_geomfromwkb</p></li>
@@ -1635,7 +1689,14 @@ per-function details are documented in the
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> try_reflect</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> try_variant_get</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> typeof</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> user</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox" checked="checked" disabled="disabled" type="checkbox"> user</p>
+<ul>
+<li><p>Spark 3.4.3 (audited 2026-05-27): <code class="docutils literal notranslate"><span class="pre">CurrentUser()</span> <span class="pre">extends</span> <span class="pre">LeafExpression</span> <span class="pre">with</span> <span class="pre">Unevaluable</span></code>; the analyzer’s <code class="docutils literal notranslate"><span class="pre">ResolveCurrentLike</span></code> rule replaces it with a <code class="docutils literal notranslate"><span class="pre">StringType</span></code> li [...]
+<li><p>Spark 3.5.8 (audited 2026-05-27): identical to 3.4.3.</p></li>
+<li><p>Spark 4.0.1 (audited 2026-05-27): identical to 3.4.3 except the resulting literal carries the default string collation.</p></li>
+<li><p>Spark 4.1.1 (audited 2026-05-27): identical to 4.0.1.</p></li>
+</ul>
+</li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> uuid</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> variant_get</p></li>
 <li class="task-list-item"><p><input class="task-list-item-checkbox" disabled="disabled" type="checkbox"> version</p></li>
diff --git a/searchindex.js b/searchindex.js
index 7baadba108..b97cdd7074 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Format Your Code": [[34, "format-your-code"]], "1. Install Comet": [[36, "install-comet"], [45, "install-comet"]], "1. Native Operators (nativeExecs map)": [[23, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[34, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[36, "clone-iceberg-and-apply-diff"]], "2. Clone Spark and Apply Diff": [[45, "clone-spark-and-apply-diff"]], "2. Sink Operators (sinks map)": [[23, "sink-operators-sinks [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Format Your Code": [[34, "format-your-code"]], "1. Install Comet": [[36, "install-comet"], [45, "install-comet"]], "1. Native Operators (nativeExecs map)": [[23, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[34, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[36, "clone-iceberg-and-apply-diff"]], "2. Clone Spark and Apply Diff": [[45, "clone-spark-and-apply-diff"]], "2. Sink Operators (sinks map)": [[23, "sink-operators-sinks [...]
\ No newline at end of file
diff --git a/user-guide/latest/compatibility/expressions/index.html b/user-guide/latest/compatibility/expressions/index.html
index 81dd1e86ba..61cc3191c2 100644
--- a/user-guide/latest/compatibility/expressions/index.html
+++ b/user-guide/latest/compatibility/expressions/index.html
@@ -572,6 +572,8 @@ the <a class="reference internal" href="../../expressions.html"><span class="std
 </li>
 <li class="toctree-l1"><a class="reference internal" href="math.html">Math Expressions</a><ul>
 <li class="toctree-l2"><a class="reference internal" href="math.html#abs">Abs</a></li>
+<li class="toctree-l2"><a class="reference internal" href="math.html#rand">Rand</a></li>
+<li class="toctree-l2"><a class="reference internal" href="math.html#randn">Randn</a></li>
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="misc.html">Miscellaneous Expressions</a><ul>
diff --git a/user-guide/latest/compatibility/expressions/math.html b/user-guide/latest/compatibility/expressions/math.html
index dea7e4f5b7..3a88373a03 100644
--- a/user-guide/latest/compatibility/expressions/math.html
+++ b/user-guide/latest/compatibility/expressions/math.html
@@ -535,6 +535,20 @@ under the License.
 <ul class="simple">
 <li><p>Only integral, floating-point, and decimal types are supported</p></li>
 </ul>
+</section>
+<section id="rand">
+<h2>Rand<a class="headerlink" href="#rand" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>The <code class="docutils literal notranslate"><span class="pre">seed</span></code> argument must be a literal value</p></li>
+</ul>
+</section>
+<section id="randn">
+<h2>Randn<a class="headerlink" href="#randn" title="Link to this heading">#</a></h2>
+<p>The following cases are not supported by Comet:</p>
+<ul class="simple">
+<li><p>The <code class="docutils literal notranslate"><span class="pre">seed</span></code> argument must be a literal value</p></li>
+</ul>
 <!--END:EXPR_COMPAT-->
 </section>
 </section>
diff --git a/user-guide/latest/compatibility/expressions/string.html b/user-guide/latest/compatibility/expressions/string.html
index 9a725475eb..d7e18d5690 100644
--- a/user-guide/latest/compatibility/expressions/string.html
+++ b/user-guide/latest/compatibility/expressions/string.html
@@ -531,7 +531,7 @@ under the License.
 <!--BEGIN:EXPR_COMPAT[string]-->
 <section id="concat">
 <h2>Concat<a class="headerlink" href="#concat" title="Link to this heading">#</a></h2>
-<p>The following incompatibilities cause <code class="docutils literal notranslate"><span class="pre">Concat</span></code> to fall back to Spark by default. Set <code class="docutils literal notranslate"><span class="pre">spark.comet.expression.Concat.allowIncompatible=true</span></code> to enable Comet acceleration despite these differences.</p>
+<p>The following cases are not supported by Comet:</p>
 <ul class="simple">
 <li><p>CONCAT supports only string input parameters</p></li>
 </ul>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org