You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/06/12 22:53:41 UTC

(datafusion-comet) branch asf-site updated: Publish built docs triggered by 787d5434684eb843110bd0a465cec8e759912588

This is an automated email from the ASF dual-hosted git repository.

github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new eae8273388 Publish built docs triggered by 787d5434684eb843110bd0a465cec8e759912588
eae8273388 is described below

commit eae82733883b4923cda5af53b857ee1688165d2e
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Jun 12 22:53:36 2026 +0000

    Publish built docs triggered by 787d5434684eb843110bd0a465cec8e759912588
---
 _sources/user-guide/latest/datatypes.md.txt | 126 ++++++++---
 _sources/user-guide/latest/operators.md.txt | 142 +++++++++---
 objects.inv                                 | Bin 3063 -> 3068 bytes
 searchindex.js                              |   2 +-
 user-guide/latest/datasources.html          |   4 +-
 user-guide/latest/datatypes.html            | 263 +++++++++++++++++++---
 user-guide/latest/expressions.html          |   4 +-
 user-guide/latest/operators.html            | 325 ++++++++++++++++++++++------
 8 files changed, 715 insertions(+), 151 deletions(-)

diff --git a/_sources/user-guide/latest/datatypes.md.txt b/_sources/user-guide/latest/datatypes.md.txt
index 7bc4d66168..80465121da 100644
--- a/_sources/user-guide/latest/datatypes.md.txt
+++ b/_sources/user-guide/latest/datatypes.md.txt
@@ -17,31 +17,101 @@
   under the License.
 -->
 
-# Supported Spark Data Types
-
-Comet supports the following Spark data types. Refer to the [Comet Compatibility Guide] for information about data
-type support in scans and other operators.
-
-[Comet Compatibility Guide]: compatibility/index.md
-
-<!-- based on org.apache.comet.serde.QueryPlanSerde.supportedDataType -->
-
-| Data Type    |
-| ------------ |
-| Null         |
-| Boolean      |
-| Byte         |
-| Short        |
-| Integer      |
-| Long         |
-| Float        |
-| Double       |
-| Decimal      |
-| String       |
-| Binary       |
-| Date         |
-| Timestamp    |
-| TimestampNTZ |
-| Struct       |
-| Array        |
-| Map          |
+# Spark Data Type Support
+
+This page is the complete reference for how Apache Comet handles each Spark data type. Comet's
+native execution path is built on Apache Arrow, so the set of types Comet can express natively
+is constrained by Arrow's type system. When a query references a type Comet does not support,
+the relevant operator falls back to Spark; results are unaffected.
+
+For per-scan and per-operator type caveats (for example, Parquet read-time conversions or
+hash-aggregate group-key restrictions), see the [Compatibility Guide](compatibility/index.md).
+
+## Status legend
+
+| Status                 | Meaning                                                                                 |
+| ---------------------- | --------------------------------------------------------------------------------------- |
+| βœ… Supported           | Native support; enabled by default.                                                     |
+| ⚠️ Supported (caveats) | Works, but with limits: certain values, contexts, or configurations fall back to Spark. |
+| πŸ”œ Planned             | Intended; tracked by an open issue or pull request.                                     |
+
+## Not currently planned
+
+The following types fall back to Spark and are not on the current roadmap. They are omitted from
+the tables below and may be reconsidered based on demand:
+
+- **`UserDefinedType`**: user-defined types are application-specific and outside the scope of native acceleration; queries referencing UDTs fall back to Spark.
+
+## Numeric
+
+| Type          | Status | Notes                                                                                                                                                     |
+| ------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `ByteType`    | βœ…     |                                                                                                                                                           |
+| `ShortType`   | βœ…     |                                                                                                                                                           |
+| `IntegerType` | βœ…     |                                                                                                                                                           |
+| `LongType`    | βœ…     |                                                                                                                                                           |
+| `FloatType`   | βœ…     | NaN and signed-zero handling can diverge from Spark in comparisons and aggregations. See [Floating-point Compatibility](compatibility/floating-point.md). |
+| `DoubleType`  | βœ…     | NaN and signed-zero handling can diverge from Spark in comparisons and aggregations. See [Floating-point Compatibility](compatibility/floating-point.md). |
+| `DecimalType` | βœ…     |                                                                                                                                                           |
+
+## String and binary
+
+| Type          | Status | Notes                                                                                                                                                         |
+| ------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `StringType`  | βœ…     | Default UTF-8 binary collation is supported. Non-default collations (Spark 4.0+) fall back ([#2190](https://github.com/apache/datafusion-comet/issues/2190)). |
+| `BinaryType`  | βœ…     |                                                                                                                                                               |
+| `CharType`    | βœ…     | Spark normalizes `CHAR(n)` to `StringType` for evaluation; same caveats apply.                                                                                |
+| `VarcharType` | βœ…     | Spark normalizes `VARCHAR(n)` to `StringType` for evaluation; same caveats apply.                                                                             |
+
+## Boolean
+
+| Type          | Status | Notes |
+| ------------- | ------ | ----- |
+| `BooleanType` | βœ…     |       |
+
+## Datetime
+
+| Type               | Status | Notes                                                                                                                                                                             |
+| ------------------ | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `DateType`         | βœ…     |                                                                                                                                                                                   |
+| `TimestampType`    | βœ…     |                                                                                                                                                                                   |
+| `TimestampNTZType` | βœ…     |                                                                                                                                                                                   |
+| `TimeType`         | ⚠️     | Spark 4.1+. Native serialization is in place; some operators (sort, shuffle, min/max) are still being wired up ([#4288](https://github.com/apache/datafusion-comet/issues/4288)). |
+
+## Interval
+
+Interval types fall back to Spark today. Native acceleration is tracked by
+[#4540](https://github.com/apache/datafusion-comet/issues/4540).
+
+| Type                    | Status | Notes             |
+| ----------------------- | ------ | ----------------- |
+| `YearMonthIntervalType` | πŸ”œ     | Tracked by #4540. |
+| `DayTimeIntervalType`   | πŸ”œ     | Tracked by #4540. |
+| `CalendarIntervalType`  | πŸ”œ     | Tracked by #4540. |
+
+## Complex
+
+| Type         | Status | Notes                                                                                                                                                                                    |
+| ------------ | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `StructType` | βœ…     | Empty structs (no fields) fall back.                                                                                                                                                     |
+| `ArrayType`  | βœ…     |                                                                                                                                                                                          |
+| `MapType`    | βœ…     | Hash aggregate group keys cannot contain a `MapType` (transitively): Arrow's row format used by DataFusion's grouped hash aggregate does not support `Map`, so such groupings fall back. |
+
+## Variant
+
+| Type          | Status | Notes                                                                                                                                                                                                          |
+| ------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `VariantType` | πŸ”œ     | Spark 4.0+. Native scan support is tracked by [#4295](https://github.com/apache/datafusion-comet/issues/4295); shredded Parquet read/write by [#3983](https://github.com/apache/datafusion-comet/issues/3983). |
+
+## Other
+
+| Type       | Status | Notes |
+| ---------- | ------ | ----- |
+| `NullType` | βœ…     |       |
+
+## See also
+
+- [Comet Compatibility Guide](compatibility/index.md) - known incompatibilities and edge cases.
+- [Parquet Scan Compatibility](compatibility/scans.md) - per-type behavior at scan time.
+- [Supported Spark Operators](operators.md) - the equivalent reference for operators.
+- [Supported Spark Expressions](expressions.md) - the equivalent reference for expressions.
diff --git a/_sources/user-guide/latest/operators.md.txt b/_sources/user-guide/latest/operators.md.txt
index d4f016f1ae..0bdf3ba919 100644
--- a/_sources/user-guide/latest/operators.md.txt
+++ b/_sources/user-guide/latest/operators.md.txt
@@ -17,33 +17,115 @@
   under the License.
 -->
 
-# Supported Spark Operators
-
-The following Spark operators are currently replaced with native versions. Query stages that contain any operators
-not supported by Comet will fall back to regular Spark execution.
-
-| Operator                          | Spark-Compatible? | Compatibility Notes                                                                                                |
-| --------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------ |
-| BatchScanExec                     | Yes               | Supports Parquet files and Apache Iceberg Parquet scans. See the [Comet Compatibility Guide] for more information. |
-| BroadcastExchangeExec             | Yes               |                                                                                                                    |
-| BroadcastHashJoinExec             | Yes               |                                                                                                                    |
-| BroadcastNestedLoopJoinExec       | Yes               | Falls back to Spark when the preserved side is broadcast (e.g. LEFT OUTER with BROADCAST on the left).             |
-| ExpandExec                        | Yes               |                                                                                                                    |
-| FileSourceScanExec                | Yes               | Supports Parquet files. See the [Comet Compatibility Guide] for more information.                                  |
-| FilterExec                        | Yes               |                                                                                                                    |
-| GenerateExec                      | Yes               | Supports `explode` and `posexplode` generators (arrays only, `_outer` variants are incompatible).                  |
-| GlobalLimitExec                   | Yes               |                                                                                                                    |
-| HashAggregateExec                 | Yes               |                                                                                                                    |
-| InsertIntoHadoopFsRelationCommand | No                | Experimental support for native Parquet writes. Disabled by default.                                               |
-| LocalLimitExec                    | Yes               |                                                                                                                    |
-| LocalTableScanExec                | No                | Experimental and disabled by default.                                                                              |
-| ObjectHashAggregateExec           | Yes               | Supports a limited number of aggregates, such as `bloom_filter_agg`.                                               |
-| ProjectExec                       | Yes               |                                                                                                                    |
-| ShuffleExchangeExec               | Yes               |                                                                                                                    |
-| ShuffledHashJoinExec              | Yes               |                                                                                                                    |
-| SortExec                          | Yes               |                                                                                                                    |
-| SortMergeJoinExec                 | Yes               |                                                                                                                    |
-| UnionExec                         | Yes               |                                                                                                                    |
-| WindowExec                        | No                | Disabled by default due to known correctness issues.                                                               |
-
-[Comet Compatibility Guide]: compatibility/operators.md
+# Spark Operator Support
+
+This page is the complete reference for how Apache Comet handles each Spark physical operator.
+Comet replaces supported operators with native equivalents. Comet runs whole subtrees of native
+operators together, so if a query stage contains an operator Comet does not support, that stage
+falls back to regular Spark execution. Results are unaffected.
+
+Operators marked βœ… Supported are enabled by default. Each can be turned off individually with
+`spark.comet.exec.OPERATOR.enabled=false` (for example `spark.comet.exec.sort.enabled=false`), and
+all native execution can be turned off with `spark.comet.exec.enabled=false`. See the
+[Comet Configuration Guide](configs.md) for the full list.
+
+## Status legend
+
+| Status                 | Meaning                                                                                                                           |
+| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
+| βœ… Supported           | Native implementation, enabled by default; works in the common case. Some inputs or forms may fall back to Spark.                 |
+| ⚠️ Supported (caveats) | Experimental or disabled by default, or accelerates only a limited subset. See the [Compatibility Guide](compatibility/index.md). |
+| πŸ”œ Planned             | Intended; tracked by an open issue or pull request.                                                                               |
+
+## Not currently planned
+
+The following operator families fall back to Spark and are not on the current roadmap. They are
+omitted from the tables below and may be reconsidered based on demand:
+
+- **Structured Streaming operators** (`StateStoreSaveExec`, `StateStoreRestoreExec`, `StreamingSymmetricHashJoinExec`, and similar): Comet targets batch execution.
+- **Cartesian / cross joins** (`CartesianProductExec`): rare and expensive, with little acceleration benefit.
+- **Sampling and range generation** (`SampleExec`, `RangeExec`): niche leaf operators.
+- **Pickled (non-Arrow) Python UDFs** (`BatchEvalPythonExec`): Comet accelerates Arrow-based Python UDFs only ([#4234](https://github.com/apache/datafusion-comet/issues/4234)).
+
+## Scans
+
+| Operator                | Status | Notes                                                                                                                                                  |
+| ----------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `FileSourceScanExec`    | βœ…     | Parquet only. Some types and configurations fall back. See [Parquet Scan Compatibility](compatibility/scans.md).                                       |
+| `BatchScanExec`         | βœ…     | Parquet, Apache Iceberg Parquet, and CSV (native) scans. See [Parquet Scan Compatibility](compatibility/scans.md) and the [Iceberg Guide](iceberg.md). |
+| `LocalTableScanExec`    | ⚠️     | Disabled by default; there is no acceleration advantage and this operator is typically only used in test code. Can be opted into via config (#4393).   |
+| `InMemoryTableScanExec` | πŸ”œ     | Cached / in-memory table scans fall back today.                                                                                                        |
+
+## Projection and filtering
+
+| Operator      | Status | Notes |
+| ------------- | ------ | ----- |
+| `ProjectExec` | βœ…     |       |
+| `FilterExec`  | βœ…     |       |
+
+## Sorting and limiting
+
+| Operator                    | Status | Notes |
+| --------------------------- | ------ | ----- |
+| `SortExec`                  | βœ…     |       |
+| `GlobalLimitExec`           | βœ…     |       |
+| `LocalLimitExec`            | βœ…     |       |
+| `CollectLimitExec`          | βœ…     |       |
+| `TakeOrderedAndProjectExec` | βœ…     |       |
+
+## Aggregation
+
+| Operator                  | Status | Notes                                                             |
+| ------------------------- | ------ | ----------------------------------------------------------------- |
+| `HashAggregateExec`       | βœ…     |                                                                   |
+| `ObjectHashAggregateExec` | βœ…     | Supports a limited set of aggregates, such as `bloom_filter_agg`. |
+| `SortAggregateExec`       | πŸ”œ     | Falls back today; Comet currently accelerates hash aggregates.    |
+
+## Joins
+
+| Operator                      | Status | Notes                                                                                                                 |
+| ----------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------- |
+| `BroadcastHashJoinExec`       | βœ…     |                                                                                                                       |
+| `ShuffledHashJoinExec`        | βœ…     |                                                                                                                       |
+| `SortMergeJoinExec`           | βœ…     |                                                                                                                       |
+| `BroadcastNestedLoopJoinExec` | βœ…     | Falls back to Spark when the preserved side is broadcast (for example LEFT OUTER with BROADCAST on the left) (#4429). |
+
+## Exchanges
+
+| Operator                | Status | Notes |
+| ----------------------- | ------ | ----- |
+| `ShuffleExchangeExec`   | βœ…     |       |
+| `BroadcastExchangeExec` | βœ…     |       |
+
+## Window
+
+| Operator               | Status | Notes                                                                                                                                            |
+| ---------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `WindowExec`           | ⚠️     | Runs natively, but only a subset of window functions is accelerated. The rest fall back. See the [expression reference](expressions.md) (#2721). |
+| `WindowGroupLimitExec` | πŸ”œ     | Window-based limit pushdown falls back today.                                                                                                    |
+
+## Generators and set operations
+
+| Operator       | Status | Notes                                                                                                                      |
+| -------------- | ------ | -------------------------------------------------------------------------------------------------------------------------- |
+| `GenerateExec` | βœ…     | Supports `explode` and `posexplode` over arrays. The `_outer` variants are incompatible, and `inline` / `stack` fall back. |
+| `ExpandExec`   | βœ…     |                                                                                                                            |
+| `UnionExec`    | βœ…     |                                                                                                                            |
+| `CoalesceExec` | βœ…     |                                                                                                                            |
+
+## Writes
+
+| Operator                 | Status | Notes                                                             |
+| ------------------------ | ------ | ----------------------------------------------------------------- |
+| `DataWritingCommandExec` | ⚠️     | Experimental native Parquet writes, disabled by default (opt-in). |
+
+## Python and UDF
+
+| Operator                                                                                | Status | Notes                                                                |
+| --------------------------------------------------------------------------------------- | ------ | -------------------------------------------------------------------- |
+| `ArrowEvalPythonExec`, `MapInArrowExec`, `MapInPandasExec`, `FlatMapGroupsInPandasExec` | πŸ”œ     | Experimental accelerated PyArrow UDF support is in progress (#4234). |
+
+## See also
+
+- [Comet Compatibility Guide](compatibility/index.md) - known incompatibilities and edge cases.
+- [Supported Spark Expressions](expressions.md) - the equivalent reference for expressions.
diff --git a/objects.inv b/objects.inv
index 3a36d70cd2..764f30197f 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/searchindex.js b/searchindex.js
index f3b4650300..589826fdc2 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!": [[49, "id1"]], "%": [[47, "id1"]], "&": [[37, "id1"]], "*": [[47, "id2"]], "+": [[47, "id3"]], "-": [[47, "id4"]], "/": [[47, "id5"]], "1. Format Your Code": [[34, "format-your-code"]], "1. Install Comet": [[55, "install-comet"], [64, "install-comet"]], "1. Native Operators (nativeExecs map)": [[23, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[34, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[55, "clone-iceberg-and-apply- [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!": [[49, "id1"]], "%": [[47, "id1"]], "&": [[37, "id1"]], "*": [[47, "id2"]], "+": [[47, "id3"]], "-": [[47, "id4"]], "/": [[47, "id5"]], "1. Format Your Code": [[34, "format-your-code"]], "1. Install Comet": [[55, "install-comet"], [64, "install-comet"]], "1. Native Operators (nativeExecs map)": [[23, "native-operators-nativeexecs-map"]], "2. Build and Verify": [[34, "build-and-verify"]], "2. Clone Iceberg and Apply Diff": [[55, "clone-iceberg-and-apply- [...]
\ No newline at end of file
diff --git a/user-guide/latest/datasources.html b/user-guide/latest/datasources.html
index eddfec3990..c653f65553 100644
--- a/user-guide/latest/datasources.html
+++ b/user-guide/latest/datasources.html
@@ -65,7 +65,7 @@ under the License.
     <script async="true" defer="true" src="https://buttons.github.io/buttons.js"></script>
     <link rel="index" title="Index" href="../../genindex.html" />
     <link rel="search" title="Search" href="../../search.html" />
-    <link rel="next" title="Supported Spark Data Types" href="datatypes.html" />
+    <link rel="next" title="Spark Data Type Support" href="datatypes.html" />
     <link rel="prev" title="Comet Configuration Settings" href="configs.html" />
   <meta name="viewport" content="width=device-width, initial-scale=1"/>
   <meta name="docsearch:language" content="en"/>
@@ -862,7 +862,7 @@ installed.</p>
        title="next page">
       <div class="prev-next-info">
         <p class="prev-next-subtitle">next</p>
-        <p class="prev-next-title">Supported Spark Data Types</p>
+        <p class="prev-next-title">Spark Data Type Support</p>
       </div>
       <i class="fa-solid fa-angle-right"></i>
     </a>
diff --git a/user-guide/latest/datatypes.html b/user-guide/latest/datatypes.html
index c395aa26dd..4edc6778ae 100644
--- a/user-guide/latest/datatypes.html
+++ b/user-guide/latest/datatypes.html
@@ -27,7 +27,7 @@ under the License.
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
 
-    <title>Supported Spark Data Types &#8212; Apache DataFusion Comet  documentation</title>
+    <title>Spark Data Type Support &#8212; Apache DataFusion Comet  documentation</title>
   
   
   
@@ -65,7 +65,7 @@ under the License.
     <script async="true" defer="true" src="https://buttons.github.io/buttons.js"></script>
     <link rel="index" title="Index" href="../../genindex.html" />
     <link rel="search" title="Search" href="../../search.html" />
-    <link rel="next" title="Supported Spark Operators" href="operators.html" />
+    <link rel="next" title="Spark Operator Support" href="operators.html" />
     <link rel="prev" title="Supported Spark Data Sources" href="datasources.html" />
   <meta name="viewport" content="width=device-width, initial-scale=1"/>
   <meta name="docsearch:language" content="en"/>
@@ -531,7 +531,7 @@ under the License.
     
     <li class="breadcrumb-item"><a href="index.html" class="nav-link">Comet 0.17.0-SNAPSHOT User Guide</a></li>
     
-    <li class="breadcrumb-item active" aria-current="page"><span class="ellipsis">Supported Spark Data Types</span></li>
+    <li class="breadcrumb-item active" aria-current="page"><span class="ellipsis">Spark Data Type Support</span></li>
   </ul>
 </nav>
 </div>
@@ -566,54 +566,263 @@ under the License.
   specific language governing permissions and limitations
   under the License.
 -->
-<section id="supported-spark-data-types">
-<h1>Supported Spark Data Types<a class="headerlink" href="#supported-spark-data-types" title="Link to this heading">#</a></h1>
-<p>Comet supports the following Spark data types. Refer to the <a class="reference internal" href="compatibility/index.html"><span class="std std-doc">Comet Compatibility Guide</span></a> for information about data
-type support in scans and other operators.</p>
-<!-- based on org.apache.comet.serde.QueryPlanSerde.supportedDataType -->
+<section id="spark-data-type-support">
+<h1>Spark Data Type Support<a class="headerlink" href="#spark-data-type-support" title="Link to this heading">#</a></h1>
+<p>This page is the complete reference for how Apache Comet handles each Spark data type. Comet’s
+native execution path is built on Apache Arrow, so the set of types Comet can express natively
+is constrained by Arrow’s type system. When a query references a type Comet does not support,
+the relevant operator falls back to Spark; results are unaffected.</p>
+<p>For per-scan and per-operator type caveats (for example, Parquet read-time conversions or
+hash-aggregate group-key restrictions), see the <a class="reference internal" href="compatibility/index.html"><span class="std std-doc">Compatibility Guide</span></a>.</p>
+<section id="status-legend">
+<h2>Status legend<a class="headerlink" href="#status-legend" title="Link to this heading">#</a></h2>
 <div class="pst-scrollable-table-container"><table class="table">
 <thead>
-<tr class="row-odd"><th class="head"><p>Data Type</p></th>
+<tr class="row-odd"><th class="head"><p>Status</p></th>
+<th class="head"><p>Meaning</p></th>
 </tr>
 </thead>
 <tbody>
-<tr class="row-even"><td><p>Null</p></td>
+<tr class="row-even"><td><p>βœ… Supported</p></td>
+<td><p>Native support; enabled by default.</p></td>
 </tr>
-<tr class="row-odd"><td><p>Boolean</p></td>
+<tr class="row-odd"><td><p>⚠️ Supported (caveats)</p></td>
+<td><p>Works, but with limits: certain values, contexts, or configurations fall back to Spark.</p></td>
 </tr>
-<tr class="row-even"><td><p>Byte</p></td>
+<tr class="row-even"><td><p>πŸ”œ Planned</p></td>
+<td><p>Intended; tracked by an open issue or pull request.</p></td>
 </tr>
-<tr class="row-odd"><td><p>Short</p></td>
+</tbody>
+</table>
+</div>
+</section>
+<section id="not-currently-planned">
+<h2>Not currently planned<a class="headerlink" href="#not-currently-planned" title="Link to this heading">#</a></h2>
+<p>The following types fall back to Spark and are not on the current roadmap. They are omitted from
+the tables below and may be reconsidered based on demand:</p>
+<ul class="simple">
+<li><p><strong><code class="docutils literal notranslate"><span class="pre">UserDefinedType</span></code></strong>: user-defined types are application-specific and outside the scope of native acceleration; queries referencing UDTs fall back to Spark.</p></li>
+</ul>
+</section>
+<section id="numeric">
+<h2>Numeric<a class="headerlink" href="#numeric" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Type</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">ByteType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>Integer</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">ShortType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>Long</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">IntegerType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">LongType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">FloatType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>NaN and signed-zero handling can diverge from Spark in comparisons and aggregations. See <a class="reference internal" href="compatibility/floating-point.html"><span class="std std-doc">Floating-point Compatibility</span></a>.</p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">DoubleType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>NaN and signed-zero handling can diverge from Spark in comparisons and aggregations. See <a class="reference internal" href="compatibility/floating-point.html"><span class="std std-doc">Floating-point Compatibility</span></a>.</p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">DecimalType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="string-and-binary">
+<h2>String and binary<a class="headerlink" href="#string-and-binary" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Type</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
 </tr>
-<tr class="row-even"><td><p>Float</p></td>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">StringType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Default UTF-8 binary collation is supported. Non-default collations (Spark 4.0+) fall back (<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/2190">#2190</a>).</p></td>
 </tr>
-<tr class="row-odd"><td><p>Double</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">BinaryType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>Decimal</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">CharType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Spark normalizes <code class="docutils literal notranslate"><span class="pre">CHAR(n)</span></code> to <code class="docutils literal notranslate"><span class="pre">StringType</span></code> for evaluation; same caveats apply.</p></td>
 </tr>
-<tr class="row-odd"><td><p>String</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">VarcharType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Spark normalizes <code class="docutils literal notranslate"><span class="pre">VARCHAR(n)</span></code> to <code class="docutils literal notranslate"><span class="pre">StringType</span></code> for evaluation; same caveats apply.</p></td>
 </tr>
-<tr class="row-even"><td><p>Binary</p></td>
+</tbody>
+</table>
+</div>
+</section>
+<section id="boolean">
+<h2>Boolean<a class="headerlink" href="#boolean" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Type</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
 </tr>
-<tr class="row-odd"><td><p>Date</p></td>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">BooleanType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>Timestamp</p></td>
+</tbody>
+</table>
+</div>
+</section>
+<section id="datetime">
+<h2>Datetime<a class="headerlink" href="#datetime" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Type</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
 </tr>
-<tr class="row-odd"><td><p>TimestampNTZ</p></td>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">DateType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>Struct</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">TimestampType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>Array</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">TimestampNTZType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>Map</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">TimeType</span></code></p></td>
+<td><p>⚠️</p></td>
+<td><p>Spark 4.1+. Native serialization is in place; some operators (sort, shuffle, min/max) are still being wired up (<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/4288">#4288</a>).</p></td>
 </tr>
 </tbody>
 </table>
 </div>
+</section>
+<section id="interval">
+<h2>Interval<a class="headerlink" href="#interval" title="Link to this heading">#</a></h2>
+<p>Interval types fall back to Spark today. Native acceleration is tracked by
+<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/4540">#4540</a>.</p>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Type</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">YearMonthIntervalType</span></code></p></td>
+<td><p>πŸ”œ</p></td>
+<td><p>Tracked by #4540.</p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">DayTimeIntervalType</span></code></p></td>
+<td><p>πŸ”œ</p></td>
+<td><p>Tracked by #4540.</p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">CalendarIntervalType</span></code></p></td>
+<td><p>πŸ”œ</p></td>
+<td><p>Tracked by #4540.</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="complex">
+<h2>Complex<a class="headerlink" href="#complex" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Type</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">StructType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Empty structs (no fields) fall back.</p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">ArrayType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">MapType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Hash aggregate group keys cannot contain a <code class="docutils literal notranslate"><span class="pre">MapType</span></code> (transitively): Arrow’s row format used by DataFusion’s grouped hash aggregate does not support <code class="docutils literal notranslate"><span class="pre">Map</span></code>, so such groupings fall back.</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="variant">
+<h2>Variant<a class="headerlink" href="#variant" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Type</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">VariantType</span></code></p></td>
+<td><p>πŸ”œ</p></td>
+<td><p>Spark 4.0+. Native scan support is tracked by <a class="reference external" href="https://github.com/apache/datafusion-comet/issues/4295">#4295</a>; shredded Parquet read/write by <a class="reference external" href="https://github.com/apache/datafusion-comet/issues/3983">#3983</a>.</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="other">
+<h2>Other<a class="headerlink" href="#other" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Type</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">NullType</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="see-also">
+<h2>See also<a class="headerlink" href="#see-also" title="Link to this heading">#</a></h2>
+<ul class="simple">
+<li><p><a class="reference internal" href="compatibility/index.html"><span class="std std-doc">Comet Compatibility Guide</span></a> - known incompatibilities and edge cases.</p></li>
+<li><p><a class="reference internal" href="compatibility/scans.html"><span class="std std-doc">Parquet Scan Compatibility</span></a> - per-type behavior at scan time.</p></li>
+<li><p><a class="reference internal" href="operators.html"><span class="std std-doc">Supported Spark Operators</span></a> - the equivalent reference for operators.</p></li>
+<li><p><a class="reference internal" href="expressions.html"><span class="std std-doc">Supported Spark Expressions</span></a> - the equivalent reference for expressions.</p></li>
+</ul>
+</section>
 </section>
 
 
@@ -640,7 +849,7 @@ type support in scans and other operators.</p>
        title="next page">
       <div class="prev-next-info">
         <p class="prev-next-subtitle">next</p>
-        <p class="prev-next-title">Supported Spark Operators</p>
+        <p class="prev-next-title">Spark Operator Support</p>
       </div>
       <i class="fa-solid fa-angle-right"></i>
     </a>
diff --git a/user-guide/latest/expressions.html b/user-guide/latest/expressions.html
index b4cc4d1e4a..81006ff147 100644
--- a/user-guide/latest/expressions.html
+++ b/user-guide/latest/expressions.html
@@ -66,7 +66,7 @@ under the License.
     <link rel="index" title="Index" href="../../genindex.html" />
     <link rel="search" title="Search" href="../../search.html" />
     <link rel="next" title="Scala UDF and Java UDF Support" href="scala_java_udfs.html" />
-    <link rel="prev" title="Supported Spark Operators" href="operators.html" />
+    <link rel="prev" title="Spark Operator Support" href="operators.html" />
   <meta name="viewport" content="width=device-width, initial-scale=1"/>
   <meta name="docsearch:language" content="en"/>
   <meta name="docsearch:version" content="" />
@@ -2745,7 +2745,7 @@ fall back to Spark.</p>
       <i class="fa-solid fa-angle-left"></i>
       <div class="prev-next-info">
         <p class="prev-next-subtitle">previous</p>
-        <p class="prev-next-title">Supported Spark Operators</p>
+        <p class="prev-next-title">Spark Operator Support</p>
       </div>
     </a>
     <a class="right-next"
diff --git a/user-guide/latest/operators.html b/user-guide/latest/operators.html
index 4461141b3e..84ea4840eb 100644
--- a/user-guide/latest/operators.html
+++ b/user-guide/latest/operators.html
@@ -27,7 +27,7 @@ under the License.
     <meta charset="utf-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
 
-    <title>Supported Spark Operators &#8212; Apache DataFusion Comet  documentation</title>
+    <title>Spark Operator Support &#8212; Apache DataFusion Comet  documentation</title>
   
   
   
@@ -66,7 +66,7 @@ under the License.
     <link rel="index" title="Index" href="../../genindex.html" />
     <link rel="search" title="Search" href="../../search.html" />
     <link rel="next" title="Spark Expression Support" href="expressions.html" />
-    <link rel="prev" title="Supported Spark Data Types" href="datatypes.html" />
+    <link rel="prev" title="Spark Data Type Support" href="datatypes.html" />
   <meta name="viewport" content="width=device-width, initial-scale=1"/>
   <meta name="docsearch:language" content="en"/>
   <meta name="docsearch:version" content="" />
@@ -531,7 +531,7 @@ under the License.
     
     <li class="breadcrumb-item"><a href="index.html" class="nav-link">Comet 0.17.0-SNAPSHOT User Guide</a></li>
     
-    <li class="breadcrumb-item active" aria-current="page"><span class="ellipsis">Supported Spark Operators</span></li>
+    <li class="breadcrumb-item active" aria-current="page"><span class="ellipsis">Spark Operator Support</span></li>
   </ul>
 </nav>
 </div>
@@ -566,105 +566,308 @@ under the License.
   specific language governing permissions and limitations
   under the License.
 -->
-<section id="supported-spark-operators">
-<h1>Supported Spark Operators<a class="headerlink" href="#supported-spark-operators" title="Link to this heading">#</a></h1>
-<p>The following Spark operators are currently replaced with native versions. Query stages that contain any operators
-not supported by Comet will fall back to regular Spark execution.</p>
+<section id="spark-operator-support">
+<h1>Spark Operator Support<a class="headerlink" href="#spark-operator-support" title="Link to this heading">#</a></h1>
+<p>This page is the complete reference for how Apache Comet handles each Spark physical operator.
+Comet replaces supported operators with native equivalents. Comet runs whole subtrees of native
+operators together, so if a query stage contains an operator Comet does not support, that stage
+falls back to regular Spark execution. Results are unaffected.</p>
+<p>Operators marked βœ… Supported are enabled by default. Each can be turned off individually with
+<code class="docutils literal notranslate"><span class="pre">spark.comet.exec.OPERATOR.enabled=false</span></code> (for example <code class="docutils literal notranslate"><span class="pre">spark.comet.exec.sort.enabled=false</span></code>), and
+all native execution can be turned off with <code class="docutils literal notranslate"><span class="pre">spark.comet.exec.enabled=false</span></code>. See the
+<a class="reference internal" href="configs.html"><span class="std std-doc">Comet Configuration Guide</span></a> for the full list.</p>
+<section id="status-legend">
+<h2>Status legend<a class="headerlink" href="#status-legend" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Status</p></th>
+<th class="head"><p>Meaning</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p>βœ… Supported</p></td>
+<td><p>Native implementation, enabled by default; works in the common case. Some inputs or forms may fall back to Spark.</p></td>
+</tr>
+<tr class="row-odd"><td><p>⚠️ Supported (caveats)</p></td>
+<td><p>Experimental or disabled by default, or accelerates only a limited subset. See the <a class="reference internal" href="compatibility/index.html"><span class="std std-doc">Compatibility Guide</span></a>.</p></td>
+</tr>
+<tr class="row-even"><td><p>πŸ”œ Planned</p></td>
+<td><p>Intended; tracked by an open issue or pull request.</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="not-currently-planned">
+<h2>Not currently planned<a class="headerlink" href="#not-currently-planned" title="Link to this heading">#</a></h2>
+<p>The following operator families fall back to Spark and are not on the current roadmap. They are
+omitted from the tables below and may be reconsidered based on demand:</p>
+<ul class="simple">
+<li><p><strong>Structured Streaming operators</strong> (<code class="docutils literal notranslate"><span class="pre">StateStoreSaveExec</span></code>, <code class="docutils literal notranslate"><span class="pre">StateStoreRestoreExec</span></code>, <code class="docutils literal notranslate"><span class="pre">StreamingSymmetricHashJoinExec</span></code>, and similar): Comet targets batch execution.</p></li>
+<li><p><strong>Cartesian / cross joins</strong> (<code class="docutils literal notranslate"><span class="pre">CartesianProductExec</span></code>): rare and expensive, with little acceleration benefit.</p></li>
+<li><p><strong>Sampling and range generation</strong> (<code class="docutils literal notranslate"><span class="pre">SampleExec</span></code>, <code class="docutils literal notranslate"><span class="pre">RangeExec</span></code>): niche leaf operators.</p></li>
+<li><p><strong>Pickled (non-Arrow) Python UDFs</strong> (<code class="docutils literal notranslate"><span class="pre">BatchEvalPythonExec</span></code>): Comet accelerates Arrow-based Python UDFs only (<a class="reference external" href="https://github.com/apache/datafusion-comet/issues/4234">#4234</a>).</p></li>
+</ul>
+</section>
+<section id="scans">
+<h2>Scans<a class="headerlink" href="#scans" title="Link to this heading">#</a></h2>
 <div class="pst-scrollable-table-container"><table class="table">
 <thead>
 <tr class="row-odd"><th class="head"><p>Operator</p></th>
-<th class="head"><p>Spark-Compatible?</p></th>
-<th class="head"><p>Compatibility Notes</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
 </tr>
 </thead>
 <tbody>
-<tr class="row-even"><td><p>BatchScanExec</p></td>
-<td><p>Yes</p></td>
-<td><p>Supports Parquet files and Apache Iceberg Parquet scans. See the <a class="reference internal" href="compatibility/operators.html"><span class="std std-doc">Comet Compatibility Guide</span></a> for more information.</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">FileSourceScanExec</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Parquet only. Some types and configurations fall back. See <a class="reference internal" href="compatibility/scans.html"><span class="std std-doc">Parquet Scan Compatibility</span></a>.</p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">BatchScanExec</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Parquet, Apache Iceberg Parquet, and CSV (native) scans. See <a class="reference internal" href="compatibility/scans.html"><span class="std std-doc">Parquet Scan Compatibility</span></a> and the <a class="reference internal" href="iceberg.html"><span class="std std-doc">Iceberg Guide</span></a>.</p></td>
 </tr>
-<tr class="row-odd"><td><p>BroadcastExchangeExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">LocalTableScanExec</span></code></p></td>
+<td><p>⚠️</p></td>
+<td><p>Disabled by default; there is no acceleration advantage and this operator is typically only used in test code. Can be opted into via config (#4393).</p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">InMemoryTableScanExec</span></code></p></td>
+<td><p>πŸ”œ</p></td>
+<td><p>Cached / in-memory table scans fall back today.</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="projection-and-filtering">
+<h2>Projection and filtering<a class="headerlink" href="#projection-and-filtering" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">ProjectExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>BroadcastHashJoinExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">FilterExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>BroadcastNestedLoopJoinExec</p></td>
-<td><p>Yes</p></td>
-<td><p>Falls back to Spark when the preserved side is broadcast (e.g. LEFT OUTER with BROADCAST on the left).</p></td>
+</tbody>
+</table>
+</div>
+</section>
+<section id="sorting-and-limiting">
+<h2>Sorting and limiting<a class="headerlink" href="#sorting-and-limiting" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
 </tr>
-<tr class="row-even"><td><p>ExpandExec</p></td>
-<td><p>Yes</p></td>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">SortExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>FileSourceScanExec</p></td>
-<td><p>Yes</p></td>
-<td><p>Supports Parquet files. See the <a class="reference internal" href="compatibility/operators.html"><span class="std std-doc">Comet Compatibility Guide</span></a> for more information.</p></td>
-</tr>
-<tr class="row-even"><td><p>FilterExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">GlobalLimitExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>GenerateExec</p></td>
-<td><p>Yes</p></td>
-<td><p>Supports <code class="docutils literal notranslate"><span class="pre">explode</span></code> and <code class="docutils literal notranslate"><span class="pre">posexplode</span></code> generators (arrays only, <code class="docutils literal notranslate"><span class="pre">_outer</span></code> variants are incompatible).</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">LocalLimitExec</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>GlobalLimitExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">CollectLimitExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>HashAggregateExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">TakeOrderedAndProjectExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>InsertIntoHadoopFsRelationCommand</p></td>
-<td><p>No</p></td>
-<td><p>Experimental support for native Parquet writes. Disabled by default.</p></td>
+</tbody>
+</table>
+</div>
+</section>
+<section id="aggregation">
+<h2>Aggregation<a class="headerlink" href="#aggregation" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
 </tr>
-<tr class="row-odd"><td><p>LocalLimitExec</p></td>
-<td><p>Yes</p></td>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">HashAggregateExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>LocalTableScanExec</p></td>
-<td><p>No</p></td>
-<td><p>Experimental and disabled by default.</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">ObjectHashAggregateExec</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Supports a limited set of aggregates, such as <code class="docutils literal notranslate"><span class="pre">bloom_filter_agg</span></code>.</p></td>
 </tr>
-<tr class="row-odd"><td><p>ObjectHashAggregateExec</p></td>
-<td><p>Yes</p></td>
-<td><p>Supports a limited number of aggregates, such as <code class="docutils literal notranslate"><span class="pre">bloom_filter_agg</span></code>.</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">SortAggregateExec</span></code></p></td>
+<td><p>πŸ”œ</p></td>
+<td><p>Falls back today; Comet currently accelerates hash aggregates.</p></td>
 </tr>
-<tr class="row-even"><td><p>ProjectExec</p></td>
-<td><p>Yes</p></td>
+</tbody>
+</table>
+</div>
+</section>
+<section id="joins">
+<h2>Joins<a class="headerlink" href="#joins" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">BroadcastHashJoinExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>ShuffleExchangeExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">ShuffledHashJoinExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>ShuffledHashJoinExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">SortMergeJoinExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>SortExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">BroadcastNestedLoopJoinExec</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Falls back to Spark when the preserved side is broadcast (for example LEFT OUTER with BROADCAST on the left) (#4429).</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="exchanges">
+<h2>Exchanges<a class="headerlink" href="#exchanges" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">ShuffleExchangeExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>SortMergeJoinExec</p></td>
-<td><p>Yes</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">BroadcastExchangeExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-odd"><td><p>UnionExec</p></td>
-<td><p>Yes</p></td>
+</tbody>
+</table>
+</div>
+</section>
+<section id="window">
+<h2>Window<a class="headerlink" href="#window" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">WindowExec</span></code></p></td>
+<td><p>⚠️</p></td>
+<td><p>Runs natively, but only a subset of window functions is accelerated. The rest fall back. See the <a class="reference internal" href="expressions.html"><span class="std std-doc">expression reference</span></a> (#2721).</p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">WindowGroupLimitExec</span></code></p></td>
+<td><p>πŸ”œ</p></td>
+<td><p>Window-based limit pushdown falls back today.</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="generators-and-set-operations">
+<h2>Generators and set operations<a class="headerlink" href="#generators-and-set-operations" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">GenerateExec</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p>Supports <code class="docutils literal notranslate"><span class="pre">explode</span></code> and <code class="docutils literal notranslate"><span class="pre">posexplode</span></code> over arrays. The <code class="docutils literal notranslate"><span class="pre">_outer</span></code> variants are incompatible, and <code class="docutils literal notranslate"><span class="pre">inline</span></code> / <code class="docutils literal notranslate"><span class="pre">stack</span></code> fall bac [...]
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">ExpandExec</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">UnionExec</span></code></p></td>
+<td><p>βœ…</p></td>
+<td><p></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">CoalesceExec</span></code></p></td>
+<td><p>βœ…</p></td>
 <td><p></p></td>
 </tr>
-<tr class="row-even"><td><p>WindowExec</p></td>
-<td><p>No</p></td>
-<td><p>Disabled by default due to known correctness issues.</p></td>
+</tbody>
+</table>
+</div>
+</section>
+<section id="writes">
+<h2>Writes<a class="headerlink" href="#writes" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">DataWritingCommandExec</span></code></p></td>
+<td><p>⚠️</p></td>
+<td><p>Experimental native Parquet writes, disabled by default (opt-in).</p></td>
 </tr>
 </tbody>
 </table>
 </div>
+</section>
+<section id="python-and-udf">
+<h2>Python and UDF<a class="headerlink" href="#python-and-udf" title="Link to this heading">#</a></h2>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Status</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">ArrowEvalPythonExec</span></code>, <code class="docutils literal notranslate"><span class="pre">MapInArrowExec</span></code>, <code class="docutils literal notranslate"><span class="pre">MapInPandasExec</span></code>, <code class="docutils literal notranslate"><span class="pre">FlatMapGroupsInPandasExec</span></code></p></td>
+<td><p>πŸ”œ</p></td>
+<td><p>Experimental accelerated PyArrow UDF support is in progress (#4234).</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+</section>
+<section id="see-also">
+<h2>See also<a class="headerlink" href="#see-also" title="Link to this heading">#</a></h2>
+<ul class="simple">
+<li><p><a class="reference internal" href="compatibility/index.html"><span class="std std-doc">Comet Compatibility Guide</span></a> - known incompatibilities and edge cases.</p></li>
+<li><p><a class="reference internal" href="expressions.html"><span class="std std-doc">Supported Spark Expressions</span></a> - the equivalent reference for expressions.</p></li>
+</ul>
+</section>
 </section>
 
 
@@ -683,7 +886,7 @@ not supported by Comet will fall back to regular Spark execution.</p>
       <i class="fa-solid fa-angle-left"></i>
       <div class="prev-next-info">
         <p class="prev-next-subtitle">previous</p>
-        <p class="prev-next-title">Supported Spark Data Types</p>
+        <p class="prev-next-title">Spark Data Type Support</p>
       </div>
     </a>
     <a class="right-next"


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org