You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/04 17:07:35 UTC
(datafusion-comet) branch asf-site updated: Publish built docs triggered by 296470cab6c3da3aec9aaf5474460b14f8a9f710
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 651d223f9f Publish built docs triggered by 296470cab6c3da3aec9aaf5474460b14f8a9f710
651d223f9f is described below
commit 651d223f9f7fc772faaa159828ad7a651a21d282
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Sep 4 17:07:31 2026 +0000
Publish built docs triggered by 296470cab6c3da3aec9aaf5474460b14f8a9f710
---
.../latest/compatibility/expressions/spark-3.4/string.md.txt | 6 ++++++
.../latest/compatibility/expressions/spark-3.5/string.md.txt | 6 ++++++
.../latest/compatibility/expressions/spark-4.0/string.md.txt | 6 ++++++
.../latest/compatibility/expressions/spark-4.1/string.md.txt | 6 ++++++
_sources/user-guide/latest/expressions.md.txt | 2 +-
searchindex.js | 2 +-
user-guide/latest/compatibility/expressions/spark-3.4/string.html | 8 ++++++++
user-guide/latest/compatibility/expressions/spark-3.5/string.html | 8 ++++++++
user-guide/latest/compatibility/expressions/spark-4.0/string.html | 8 ++++++++
user-guide/latest/compatibility/expressions/spark-4.1/string.html | 8 ++++++++
user-guide/latest/expressions.html | 4 ++--
11 files changed, 60 insertions(+), 4 deletions(-)
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/string.md.txt
index c7149dc67f..8adb125749 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.4/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.4/string.md.txt
@@ -37,6 +37,12 @@ The following cases have no native implementation and always run in the JVM usin
- CONCAT supports only string input parameters
+## ConcatWs
+
+The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
+
+- `concat_ws` with `array<string>` arguments: Spark flattens the array elements into the strings to join, which DataFusion's `concat_ws` does not support (https://github.com/apache/datafusion-comet/issues/5675)
+
## Contains
The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/string.md.txt
index c7149dc67f..8adb125749 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-3.5/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-3.5/string.md.txt
@@ -37,6 +37,12 @@ The following cases have no native implementation and always run in the JVM usin
- CONCAT supports only string input parameters
+## ConcatWs
+
+The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
+
+- `concat_ws` with `array<string>` arguments: Spark flattens the array elements into the strings to join, which DataFusion's `concat_ws` does not support (https://github.com/apache/datafusion-comet/issues/5675)
+
## Contains
The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/string.md.txt
index f73a41dd90..5d8f958f09 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.0/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.0/string.md.txt
@@ -37,6 +37,12 @@ The following cases have no native implementation and always run in the JVM usin
- CONCAT supports only string input parameters
+## ConcatWs
+
+The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
+
+- `concat_ws` with `array<string>` arguments: Spark flattens the array elements into the strings to join, which DataFusion's `concat_ws` does not support (https://github.com/apache/datafusion-comet/issues/5675)
+
## Contains
The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
diff --git a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/string.md.txt b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/string.md.txt
index f73a41dd90..5d8f958f09 100644
--- a/_sources/user-guide/latest/compatibility/expressions/spark-4.1/string.md.txt
+++ b/_sources/user-guide/latest/compatibility/expressions/spark-4.1/string.md.txt
@@ -37,6 +37,12 @@ The following cases have no native implementation and always run in the JVM usin
- CONCAT supports only string input parameters
+## ConcatWs
+
+The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
+
+- `concat_ws` with `array<string>` arguments: Spark flattens the array elements into the strings to join, which DataFusion's `concat_ws` does not support (https://github.com/apache/datafusion-comet/issues/5675)
+
## Contains
The following cases have no native implementation and always run in the JVM using Spark's code-generated implementation (inside the Comet pipeline):
diff --git a/_sources/user-guide/latest/expressions.md.txt b/_sources/user-guide/latest/expressions.md.txt
index 73926b8972..ee25170ae3 100644
--- a/_sources/user-guide/latest/expressions.md.txt
+++ b/_sources/user-guide/latest/expressions.md.txt
@@ -561,7 +561,7 @@ The type-name conversion functions (`bigint`, `binary`, `boolean`, `date`, `deci
| `chr` | ✅ | Native | |
| `collate` | 🔜 | — | Spark collation (umbrella [#2190](https://github.com/apache/datafusion-comet/issues/2190)) |
| `collation` | ✅ | — | Constant-folded to a literal (Spark 4.0+) |
-| `concat_ws` | ✅ | Native | |
+| `concat_ws` | ✅ | Hybrid | Array arguments route through the JVM codegen dispatcher; string arguments run natively ([details](compatibility/expressions/string.md)) |
| `contains` | ✅ | — | |
| `decode` | ✅ | — | |
| `elt` | ✅ | Codegen dispatch | |
diff --git a/searchindex.js b/searchindex.js
index 348c9abcfa..04a27fa21e 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[55, "id1"]], "%": [[53, "id1"]], "&": [[43, "id1"]], "*": [[53, "id2"]], "+": [[53, "id3"]], "-": [[53, "id4"]], "/": [[53, "id5"]], "1. Format Your Code": [[40, "format-your-code"]], "1. Install Comet": [[61, "install-comet"], [71, "install-comet"]], "1. Native Operators (nativeExecs map)": [[28, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[40, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[61, "clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[55, "id1"]], "%": [[53, "id1"]], "&": [[43, "id1"]], "*": [[53, "id2"]], "+": [[53, "id3"]], "-": [[53, "id4"]], "/": [[53, "id5"]], "1. Format Your Code": [[40, "format-your-code"]], "1. Install Comet": [[61, "install-comet"], [71, "install-comet"]], "1. Native Operators (nativeExecs map)": [[28, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[40, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[61, "clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/latest/compatibility/expressions/spark-3.4/string.html b/user-guide/latest/compatibility/expressions/spark-3.4/string.html
index 83f837dfc6..e321419c6d 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.4/string.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.4/string.html
@@ -801,6 +801,13 @@ under the License.
<li><p>CONCAT supports only string input parameters</p></li>
</ul>
</section>
+<section id="concatws">
+<h2>ConcatWs<a class="headerlink" href="#concatws" title="Link to this heading">#</a></h2>
+<p>The following cases have no native implementation and always run in the JVM using Spark’s code-generated implementation (inside the Comet pipeline):</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">concat_ws</span></code> with <code class="docutils literal notranslate"><span class="pre">array<string></span></code> arguments: Spark flattens the array elements into the strings to join, which DataFusion’s <code class="docutils literal notranslate"><span class="pre">concat_ws</span></code> does not support (https://github.com/apache/datafusion-comet/issues/5675)</p></li>
+</ul>
+</section>
<section id="contains">
<h2>Contains<a class="headerlink" href="#contains" title="Link to this heading">#</a></h2>
<p>The following cases have no native implementation and always run in the JVM using Spark’s code-generated implementation (inside the Comet pipeline):</p>
@@ -1006,6 +1013,7 @@ under the License.
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#bitlength">BitLength</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#concat">Concat</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#concatws">ConcatWs</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#contains">Contains</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#endswith">EndsWith</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#getjsonobject">GetJsonObject</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-3.5/string.html b/user-guide/latest/compatibility/expressions/spark-3.5/string.html
index 33d859e84c..9af6688cbd 100644
--- a/user-guide/latest/compatibility/expressions/spark-3.5/string.html
+++ b/user-guide/latest/compatibility/expressions/spark-3.5/string.html
@@ -801,6 +801,13 @@ under the License.
<li><p>CONCAT supports only string input parameters</p></li>
</ul>
</section>
+<section id="concatws">
+<h2>ConcatWs<a class="headerlink" href="#concatws" title="Link to this heading">#</a></h2>
+<p>The following cases have no native implementation and always run in the JVM using Spark’s code-generated implementation (inside the Comet pipeline):</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">concat_ws</span></code> with <code class="docutils literal notranslate"><span class="pre">array<string></span></code> arguments: Spark flattens the array elements into the strings to join, which DataFusion’s <code class="docutils literal notranslate"><span class="pre">concat_ws</span></code> does not support (https://github.com/apache/datafusion-comet/issues/5675)</p></li>
+</ul>
+</section>
<section id="contains">
<h2>Contains<a class="headerlink" href="#contains" title="Link to this heading">#</a></h2>
<p>The following cases have no native implementation and always run in the JVM using Spark’s code-generated implementation (inside the Comet pipeline):</p>
@@ -1006,6 +1013,7 @@ under the License.
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#bitlength">BitLength</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#concat">Concat</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#concatws">ConcatWs</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#contains">Contains</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#endswith">EndsWith</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#getjsonobject">GetJsonObject</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.0/string.html b/user-guide/latest/compatibility/expressions/spark-4.0/string.html
index 930d652980..0defde8963 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.0/string.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.0/string.html
@@ -801,6 +801,13 @@ under the License.
<li><p>CONCAT supports only string input parameters</p></li>
</ul>
</section>
+<section id="concatws">
+<h2>ConcatWs<a class="headerlink" href="#concatws" title="Link to this heading">#</a></h2>
+<p>The following cases have no native implementation and always run in the JVM using Spark’s code-generated implementation (inside the Comet pipeline):</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">concat_ws</span></code> with <code class="docutils literal notranslate"><span class="pre">array<string></span></code> arguments: Spark flattens the array elements into the strings to join, which DataFusion’s <code class="docutils literal notranslate"><span class="pre">concat_ws</span></code> does not support (https://github.com/apache/datafusion-comet/issues/5675)</p></li>
+</ul>
+</section>
<section id="contains">
<h2>Contains<a class="headerlink" href="#contains" title="Link to this heading">#</a></h2>
<p>The following cases have no native implementation and always run in the JVM using Spark’s code-generated implementation (inside the Comet pipeline):</p>
@@ -1015,6 +1022,7 @@ under the License.
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#bitlength">BitLength</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#concat">Concat</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#concatws">ConcatWs</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#contains">Contains</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#endswith">EndsWith</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#getjsonobject">GetJsonObject</a></li>
diff --git a/user-guide/latest/compatibility/expressions/spark-4.1/string.html b/user-guide/latest/compatibility/expressions/spark-4.1/string.html
index 732b87b725..3d80edb05e 100644
--- a/user-guide/latest/compatibility/expressions/spark-4.1/string.html
+++ b/user-guide/latest/compatibility/expressions/spark-4.1/string.html
@@ -801,6 +801,13 @@ under the License.
<li><p>CONCAT supports only string input parameters</p></li>
</ul>
</section>
+<section id="concatws">
+<h2>ConcatWs<a class="headerlink" href="#concatws" title="Link to this heading">#</a></h2>
+<p>The following cases have no native implementation and always run in the JVM using Spark’s code-generated implementation (inside the Comet pipeline):</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">concat_ws</span></code> with <code class="docutils literal notranslate"><span class="pre">array<string></span></code> arguments: Spark flattens the array elements into the strings to join, which DataFusion’s <code class="docutils literal notranslate"><span class="pre">concat_ws</span></code> does not support (https://github.com/apache/datafusion-comet/issues/5675)</p></li>
+</ul>
+</section>
<section id="contains">
<h2>Contains<a class="headerlink" href="#contains" title="Link to this heading">#</a></h2>
<p>The following cases have no native implementation and always run in the JVM using Spark’s code-generated implementation (inside the Comet pipeline):</p>
@@ -1015,6 +1022,7 @@ under the License.
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#bitlength">BitLength</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#concat">Concat</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#concatws">ConcatWs</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#contains">Contains</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#endswith">EndsWith</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#getjsonobject">GetJsonObject</a></li>
diff --git a/user-guide/latest/expressions.html b/user-guide/latest/expressions.html
index cb89ea400f..d25a2a818a 100644
--- a/user-guide/latest/expressions.html
+++ b/user-guide/latest/expressions.html
@@ -2937,8 +2937,8 @@ expressions. The following function families are <strong>not currently planned</
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">concat_ws</span></code></p></td>
<td><p>✅</p></td>
-<td><p>Native</p></td>
-<td><p></p></td>
+<td><p>Hybrid</p></td>
+<td><p>Array arguments route through the JVM codegen dispatcher; string arguments run natively (<a class="reference internal" href="#compatibility/expressions/string.md"><span class="xref myst">details</span></a>)</p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">contains</span></code></p></td>
<td><p>✅</p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org