You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2024/12/06 20:05:22 UTC
(datafusion-comet) branch asf-site updated: Publish built docs triggered by b95dc1d84554b7cd5728ed53a24463a2e111ba44
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d9ccecb0 Publish built docs triggered by b95dc1d84554b7cd5728ed53a24463a2e111ba44
d9ccecb0 is described below
commit d9ccecb042cffdf731850acd6029ad161a426838
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Dec 6 20:05:19 2024 +0000
Publish built docs triggered by b95dc1d84554b7cd5728ed53a24463a2e111ba44
---
_sources/contributor-guide/plugin_overview.md.txt | 59 +++++---
contributor-guide/adding_a_new_expression.html | 2 +-
contributor-guide/benchmark-results/tpc-ds.html | 2 +-
contributor-guide/benchmark-results/tpc-h.html | 2 +-
contributor-guide/benchmarking.html | 2 +-
contributor-guide/contributing.html | 2 +-
contributor-guide/debugging.html | 2 +-
contributor-guide/development.html | 2 +-
contributor-guide/plugin_overview.html | 155 ++++++++++++++--------
contributor-guide/profiling_native_code.html | 2 +-
contributor-guide/spark-sql-tests.html | 2 +-
genindex.html | 2 +-
index.html | 5 +-
search.html | 2 +-
searchindex.js | 2 +-
user-guide/compatibility.html | 2 +-
user-guide/configs.html | 2 +-
user-guide/datasources.html | 2 +-
user-guide/datatypes.html | 2 +-
user-guide/expressions.html | 2 +-
user-guide/installation.html | 2 +-
user-guide/kubernetes.html | 2 +-
user-guide/operators.html | 2 +-
user-guide/overview.html | 2 +-
user-guide/source.html | 2 +-
user-guide/tuning.html | 2 +-
26 files changed, 168 insertions(+), 97 deletions(-)
diff --git a/_sources/contributor-guide/plugin_overview.md.txt b/_sources/contributor-guide/plugin_overview.md.txt
index 3e7c24f5..add4172d 100644
--- a/_sources/contributor-guide/plugin_overview.md.txt
+++ b/_sources/contributor-guide/plugin_overview.md.txt
@@ -19,17 +19,26 @@ under the License.
# Comet Plugin Architecture
+## Overview
+
+The Comet plugin enhances Spark SQL by introducing optimized query execution and shuffle mechanisms leveraging
+native code. It integrates with Spark's plugin framework and extension API to replace or extend Spark's
+default behavior.
+
+---
+
+# Plugin Components
+
## Comet SQL Plugin
-The entry point to Comet is the `org.apache.spark.CometPlugin` class, which can be registered with Spark by adding the
-following setting to the Spark configuration when launching `spark-shell` or `spark-submit`:
+The entry point to Comet is the org.apache.spark.CometPlugin class, which is registered in Spark using the following
+configuration:
```
--conf spark.plugins=org.apache.spark.CometPlugin
```
-This class is loaded by Spark's plugin framework. It will be instantiated in the Spark driver only. Comet does not
-provide any executor plugins.
+The plugin is loaded on the Spark driver and does not provide executor-side plugins.
The plugin will update the current `SparkConf` with the extra configuration provided by Comet, such as executor memory
configuration.
@@ -87,7 +96,7 @@ In the native code there is a `PhysicalPlanner` struct (in `planner.rs`) which c
Apache DataFusion `ExecutionPlan`. In some cases, Comet provides specialized physical operators and expressions to
override the DataFusion versions to ensure compatibility with Apache Spark.
-The leaf nodes in the physical plan are always `ScanExec` and each of these operators will make a JNI call to
+The leaf nodes in the physical plan are always `ScanExec` and each of these operators will make a JNI call to
`CometBatchIterator.next()` to fetch the next input batch. The input could be a Comet native Parquet scan,
a Spark exchange, or another native plan.
@@ -95,15 +104,38 @@ a Spark exchange, or another native plan.
partitions. Each call to `CometExecIterator.next()` will invoke `Native.executePlan`. Once the plan finishes
executing, the resulting Arrow batches are imported into the JVM using Arrow FFI.
-## Arrow
+## Shuffle
-Due to the hybrid execution model, it is necessary to pass batches of data between the JVM and native code.
+Comet integrates with Spark's shuffle mechanism, optimizing both shuffle writes and reads. Comet's shuffle manager
+must be registered with Spark using the following configuration:
+
+```
+--conf spark.shuffle.manager=org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager
+```
+
+### Shuffle Writes
-Comet uses a combination of Arrow FFI and Arrow IPC to achieve this.
+For shuffle writes, a `ShuffleMapTask` runs in the executors. This task contains a `ShuffleDependency` that is
+broadcast to all of the executors. It then passes the input RDD to `ShuffleWriteProcessor.write()` which
+requests a `ShuffleWriter` from the shuffle manager, and this is where it gets a Comet shuffle writer.
-### Arrow FFI
+`ShuffleWriteProcessor` then invokes the dependency RDD and fetches rows/batches and passes them to Comet's
+shuffle writer, which writes batches to disk in Arrow IPC format.
-The foundation for Arrow FFI is the [Arrow C Data Interface], which provides a stable ABI-compatible interface for
+As a result, we cannot avoid having one native plan to produce the shuffle input and another native plan for
+writing the batches to the shuffle file.
+
+### Shuffle Reads
+
+For shuffle reads a `ShuffledRDD` requests a `ShuffleReader` from the shuffle manager. Comet provides a
+`CometBlockStoreShuffleReader` which is implemented in JVM and fetches blocks from Spark and then creates an
+`ArrowReaderIterator` to process the blocks using Arrow's `StreamReader` for decoding IPC batches.
+
+## Arrow FFI
+
+Due to the hybrid execution model, it is necessary to pass batches of data between the JVM and native code.
+
+The foundation for Arrow FFI is the [Arrow C Data Interface], which provides a stable ABI-compatible interface for
accessing Arrow data structures from multiple languages.
[Arrow C Data Interface]: https://arrow.apache.org/docs/format/CDataInterface.html
@@ -111,13 +143,6 @@ accessing Arrow data structures from multiple languages.
- `CometExecIterator` invokes native plans and uses Arrow FFI to read the output batches
- Native `ScanExec` operators call `CometBatchIterator` via JNI to fetch input batches from the JVM
-### Arrow IPC
-
-Comet native shuffle uses Arrow IPC to write batches to the shuffle files.
-
-- `CometShuffleWriteProcessor` invokes a native plan to fetch batches and then passes them to native `ShuffleWriterExec`
-- `CometBlockStoreShuffleReader` reads batches from shuffle files
-
## End to End Flow
The following diagram shows an example of the end-to-end flow for a query stage.
diff --git a/contributor-guide/adding_a_new_expression.html b/contributor-guide/adding_a_new_expression.html
index 0699189c..c563c421 100644
--- a/contributor-guide/adding_a_new_expression.html
+++ b/contributor-guide/adding_a_new_expression.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/contributor-guide/benchmark-results/tpc-ds.html b/contributor-guide/benchmark-results/tpc-ds.html
index 9e3936f3..e553aee3 100644
--- a/contributor-guide/benchmark-results/tpc-ds.html
+++ b/contributor-guide/benchmark-results/tpc-ds.html
@@ -175,7 +175,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/contributor-guide/benchmark-results/tpc-h.html b/contributor-guide/benchmark-results/tpc-h.html
index e4e07736..bf15717f 100644
--- a/contributor-guide/benchmark-results/tpc-h.html
+++ b/contributor-guide/benchmark-results/tpc-h.html
@@ -175,7 +175,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/contributor-guide/benchmarking.html b/contributor-guide/benchmarking.html
index edc7699b..64a77feb 100644
--- a/contributor-guide/benchmarking.html
+++ b/contributor-guide/benchmarking.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/contributor-guide/contributing.html b/contributor-guide/contributing.html
index 3eefa9ff..c599fe37 100644
--- a/contributor-guide/contributing.html
+++ b/contributor-guide/contributing.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/contributor-guide/debugging.html b/contributor-guide/debugging.html
index 7c06bf90..9145151d 100644
--- a/contributor-guide/debugging.html
+++ b/contributor-guide/debugging.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/contributor-guide/development.html b/contributor-guide/development.html
index aa7440e5..6f1ed9a9 100644
--- a/contributor-guide/development.html
+++ b/contributor-guide/development.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1 current active">
diff --git a/contributor-guide/plugin_overview.html b/contributor-guide/plugin_overview.html
index edb1d7a2..e30df4f4 100644
--- a/contributor-guide/plugin_overview.html
+++ b/contributor-guide/plugin_overview.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1 current active">
<a class="current reference internal" href="#">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
@@ -281,55 +281,79 @@ under the License.
<nav id="bd-toc-nav">
<ul class="visible nav section-nav flex-column">
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#comet-sql-plugin">
- Comet SQL Plugin
- </a>
+ <li class="toc-h1 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#">
+ Comet Plugin Architecture
+ </a>
+ <ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#overview">
+ Overview
+ </a>
+ </li>
+ </ul>
</li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#cometsparksessionextensions">
- CometSparkSessionExtensions
- </a>
- <ul class="nav section-nav flex-column">
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#cometscanrule">
- CometScanRule
+ <li class="toc-h1 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#plugin-components">
+ Plugin Components
+ </a>
+ <ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#comet-sql-plugin">
+ Comet SQL Plugin
</a>
</li>
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#cometexecrule">
- CometExecRule
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#cometsparksessionextensions">
+ CometSparkSessionExtensions
</a>
+ <ul class="nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#cometscanrule">
+ CometScanRule
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#cometexecrule">
+ CometExecRule
+ </a>
+ </li>
+ </ul>
</li>
- </ul>
- </li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#query-execution">
- Query Execution
- </a>
- </li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#arrow">
- Arrow
- </a>
- <ul class="nav section-nav flex-column">
- <li class="toc-h3 nav-item toc-entry">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#query-execution">
+ Query Execution
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#shuffle">
+ Shuffle
+ </a>
+ <ul class="nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#shuffle-writes">
+ Shuffle Writes
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#shuffle-reads">
+ Shuffle Reads
+ </a>
+ </li>
+ </ul>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#arrow-ffi">
Arrow FFI
</a>
</li>
- <li class="toc-h3 nav-item toc-entry">
- <a class="reference internal nav-link" href="#arrow-ipc">
- Arrow IPC
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#end-to-end-flow">
+ End to End Flow
</a>
</li>
</ul>
</li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#end-to-end-flow">
- End to End Flow
- </a>
- </li>
</ul>
</nav>
@@ -378,15 +402,24 @@ under the License.
-->
<section id="comet-plugin-architecture">
<h1>Comet Plugin Architecture<a class="headerlink" href="#comet-plugin-architecture" title="Link to this heading">¶</a></h1>
+<section id="overview">
+<h2>Overview<a class="headerlink" href="#overview" title="Link to this heading">¶</a></h2>
+<p>The Comet plugin enhances Spark SQL by introducing optimized query execution and shuffle mechanisms leveraging
+native code. It integrates with Spark’s plugin framework and extension API to replace or extend Spark’s
+default behavior.</p>
+</section>
+</section>
+<hr class="docutils" />
+<section id="plugin-components">
+<h1>Plugin Components<a class="headerlink" href="#plugin-components" title="Link to this heading">¶</a></h1>
<section id="comet-sql-plugin">
<h2>Comet SQL Plugin<a class="headerlink" href="#comet-sql-plugin" title="Link to this heading">¶</a></h2>
-<p>The entry point to Comet is the <code class="docutils literal notranslate"><span class="pre">org.apache.spark.CometPlugin</span></code> class, which can be registered with Spark by adding the
-following setting to the Spark configuration when launching <code class="docutils literal notranslate"><span class="pre">spark-shell</span></code> or <code class="docutils literal notranslate"><span class="pre">spark-submit</span></code>:</p>
+<p>The entry point to Comet is the org.apache.spark.CometPlugin class, which is registered in Spark using the following
+configuration:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">--</span><span class="n">conf</span> <span class="n">spark</span><span class="o">.</span><span class="n">plugins</span><span class="o">=</span><span class="n">org</span><span class="o">.</span><span class="n">apache</span><span class="o">.</span><span class="n">spark</span><span class="o">.</span><span class="n">CometPlugin</span>
</pre></div>
</div>
-<p>This class is loaded by Spark’s plugin framework. It will be instantiated in the Spark driver only. Comet does not
-provide any executor plugins.</p>
+<p>The plugin is loaded on the Spark driver and does not provide executor-side plugins.</p>
<p>The plugin will update the current <code class="docutils literal notranslate"><span class="pre">SparkConf</span></code> with the extra configuration provided by Comet, such as executor memory
configuration.</p>
<p>The plugin also registers <code class="docutils literal notranslate"><span class="pre">CometSparkSessionExtensions</span></code> with Spark’s extension API.</p>
@@ -440,12 +473,33 @@ a Spark exchange, or another native plan.</p>
partitions. Each call to <code class="docutils literal notranslate"><span class="pre">CometExecIterator.next()</span></code> will invoke <code class="docutils literal notranslate"><span class="pre">Native.executePlan</span></code>. Once the plan finishes
executing, the resulting Arrow batches are imported into the JVM using Arrow FFI.</p>
</section>
-<section id="arrow">
-<h2>Arrow<a class="headerlink" href="#arrow" title="Link to this heading">¶</a></h2>
-<p>Due to the hybrid execution model, it is necessary to pass batches of data between the JVM and native code.</p>
-<p>Comet uses a combination of Arrow FFI and Arrow IPC to achieve this.</p>
+<section id="shuffle">
+<h2>Shuffle<a class="headerlink" href="#shuffle" title="Link to this heading">¶</a></h2>
+<p>Comet integrates with Spark’s shuffle mechanism, optimizing both shuffle writes and reads. Comet’s shuffle manager
+must be registered with Spark using the following configuration:</p>
+<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">--</span><span class="n">conf</span> <span class="n">spark</span><span class="o">.</span><span class="n">shuffle</span><span class="o">.</span><span class="n">manager</span><span class="o">=</span><span class="n">org</span><span class="o">.</span><span class="n">apache</span><span class="o">.</span><span class="n">spark</span><span class="o">.</span><span class="n">sql</span><span class="o [...]
+</pre></div>
+</div>
+<section id="shuffle-writes">
+<h3>Shuffle Writes<a class="headerlink" href="#shuffle-writes" title="Link to this heading">¶</a></h3>
+<p>For shuffle writes, a <code class="docutils literal notranslate"><span class="pre">ShuffleMapTask</span></code> runs in the executors. This task contains a <code class="docutils literal notranslate"><span class="pre">ShuffleDependency</span></code> that is
+broadcast to all of the executors. It then passes the input RDD to <code class="docutils literal notranslate"><span class="pre">ShuffleWriteProcessor.write()</span></code> which
+requests a <code class="docutils literal notranslate"><span class="pre">ShuffleWriter</span></code> from the shuffle manager, and this is where it gets a Comet shuffle writer.</p>
+<p><code class="docutils literal notranslate"><span class="pre">ShuffleWriteProcessor</span></code> then invokes the dependency RDD and fetches rows/batches and passes them to Comet’s
+shuffle writer, which writes batches to disk in Arrow IPC format.</p>
+<p>As a result, we cannot avoid having one native plan to produce the shuffle input and another native plan for
+writing the batches to the shuffle file.</p>
+</section>
+<section id="shuffle-reads">
+<h3>Shuffle Reads<a class="headerlink" href="#shuffle-reads" title="Link to this heading">¶</a></h3>
+<p>For shuffle reads a <code class="docutils literal notranslate"><span class="pre">ShuffledRDD</span></code> requests a <code class="docutils literal notranslate"><span class="pre">ShuffleReader</span></code> from the shuffle manager. Comet provides a
+<code class="docutils literal notranslate"><span class="pre">CometBlockStoreShuffleReader</span></code> which is implemented in JVM and fetches blocks from Spark and then creates an
+<code class="docutils literal notranslate"><span class="pre">ArrowReaderIterator</span></code> to process the blocks using Arrow’s <code class="docutils literal notranslate"><span class="pre">StreamReader</span></code> for decoding IPC batches.</p>
+</section>
+</section>
<section id="arrow-ffi">
-<h3>Arrow FFI<a class="headerlink" href="#arrow-ffi" title="Link to this heading">¶</a></h3>
+<h2>Arrow FFI<a class="headerlink" href="#arrow-ffi" title="Link to this heading">¶</a></h2>
+<p>Due to the hybrid execution model, it is necessary to pass batches of data between the JVM and native code.</p>
<p>The foundation for Arrow FFI is the <a class="reference external" href="https://arrow.apache.org/docs/format/CDataInterface.html">Arrow C Data Interface</a>, which provides a stable ABI-compatible interface for
accessing Arrow data structures from multiple languages.</p>
<ul class="simple">
@@ -453,15 +507,6 @@ accessing Arrow data structures from multiple languages.</p>
<li><p>Native <code class="docutils literal notranslate"><span class="pre">ScanExec</span></code> operators call <code class="docutils literal notranslate"><span class="pre">CometBatchIterator</span></code> via JNI to fetch input batches from the JVM</p></li>
</ul>
</section>
-<section id="arrow-ipc">
-<h3>Arrow IPC<a class="headerlink" href="#arrow-ipc" title="Link to this heading">¶</a></h3>
-<p>Comet native shuffle uses Arrow IPC to write batches to the shuffle files.</p>
-<ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">CometShuffleWriteProcessor</span></code> invokes a native plan to fetch batches and then passes them to native <code class="docutils literal notranslate"><span class="pre">ShuffleWriterExec</span></code></p></li>
-<li><p><code class="docutils literal notranslate"><span class="pre">CometBlockStoreShuffleReader</span></code> reads batches from shuffle files</p></li>
-</ul>
-</section>
-</section>
<section id="end-to-end-flow">
<h2>End to End Flow<a class="headerlink" href="#end-to-end-flow" title="Link to this heading">¶</a></h2>
<p>The following diagram shows an example of the end-to-end flow for a query stage.</p>
diff --git a/contributor-guide/profiling_native_code.html b/contributor-guide/profiling_native_code.html
index d538c345..75ad7838 100644
--- a/contributor-guide/profiling_native_code.html
+++ b/contributor-guide/profiling_native_code.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/contributor-guide/spark-sql-tests.html b/contributor-guide/spark-sql-tests.html
index 0ca76eac..76083a50 100644
--- a/contributor-guide/spark-sql-tests.html
+++ b/contributor-guide/spark-sql-tests.html
@@ -176,7 +176,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/genindex.html b/genindex.html
index aee84d66..ed430752 100644
--- a/genindex.html
+++ b/genindex.html
@@ -174,7 +174,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/index.html b/index.html
index e681db74..4cc411d8 100644
--- a/index.html
+++ b/index.html
@@ -176,7 +176,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
@@ -333,7 +333,8 @@ as a native runtime to achieve improvement in terms of query efficiency and quer
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Contributor Guide</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/contributing.html">Getting Started</a></li>
-<li class="toctree-l1"><a class="reference internal" href="contributor-guide/plugin_overview.html">Comet Plugin Overview</a></li>
+<li class="toctree-l1"><a class="reference internal" href="contributor-guide/plugin_overview.html">Comet Plugin Architecture</a></li>
+<li class="toctree-l1"><a class="reference internal" href="contributor-guide/plugin_overview.html#plugin-components">Plugin Components</a></li>
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/development.html">Development Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/debugging.html">Debugging Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/benchmarking.html">Benchmarking Guide</a></li>
diff --git a/search.html b/search.html
index 7cb76e4b..c6d0b897 100644
--- a/search.html
+++ b/search.html
@@ -181,7 +181,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/searchindex.js b/searchindex.js
index f598d9a7..74291463 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Install Comet": [[9, "install-comet"]], "2. Clone Spark and Apply Diff": [[9, "clone-spark-and-apply-diff"]], "3. Run Spark SQL Tests": [[9, "run-spark-sql-tests"]], "ANSI mode": [[11, "ansi-mode"]], "API Differences Between Spark Versions": [[0, "api-differences-between-spark-versions"]], "ASF Links": [[10, null]], "Adding Spark-side Tests for the New Expression": [[0, "adding-spark-side-tests-for-the-new-expression"]], "Adding a New Expression": [[0, [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Install Comet": [[9, "install-comet"]], "2. Clone Spark and Apply Diff": [[9, "clone-spark-and-apply-diff"]], "3. Run Spark SQL Tests": [[9, "run-spark-sql-tests"]], "ANSI mode": [[11, "ansi-mode"]], "API Differences Between Spark Versions": [[0, "api-differences-between-spark-versions"]], "ASF Links": [[10, null]], "Adding Spark-side Tests for the New Expression": [[0, "adding-spark-side-tests-for-the-new-expression"]], "Adding a New Expression": [[0, [...]
\ No newline at end of file
diff --git a/user-guide/compatibility.html b/user-guide/compatibility.html
index a507ea84..4308e239 100644
--- a/user-guide/compatibility.html
+++ b/user-guide/compatibility.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/configs.html b/user-guide/configs.html
index 21776947..6ad8cb8c 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/datasources.html b/user-guide/datasources.html
index 9de7b14c..ded9c996 100644
--- a/user-guide/datasources.html
+++ b/user-guide/datasources.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/datatypes.html b/user-guide/datatypes.html
index 88265922..11b0746f 100644
--- a/user-guide/datatypes.html
+++ b/user-guide/datatypes.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/expressions.html b/user-guide/expressions.html
index f56c2521..0a7409bd 100644
--- a/user-guide/expressions.html
+++ b/user-guide/expressions.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/installation.html b/user-guide/installation.html
index 7dba8c18..5f01bf26 100644
--- a/user-guide/installation.html
+++ b/user-guide/installation.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/kubernetes.html b/user-guide/kubernetes.html
index 192212bf..511e3111 100644
--- a/user-guide/kubernetes.html
+++ b/user-guide/kubernetes.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/operators.html b/user-guide/operators.html
index c127ade3..d7420218 100644
--- a/user-guide/operators.html
+++ b/user-guide/operators.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/overview.html b/user-guide/overview.html
index 98af6825..ed7e7b82 100644
--- a/user-guide/overview.html
+++ b/user-guide/overview.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/source.html b/user-guide/source.html
index 82f7d2fd..436c0ef3 100644
--- a/user-guide/source.html
+++ b/user-guide/source.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
diff --git a/user-guide/tuning.html b/user-guide/tuning.html
index 7ffdadba..7032944d 100644
--- a/user-guide/tuning.html
+++ b/user-guide/tuning.html
@@ -177,7 +177,7 @@ under the License.
</li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/plugin_overview.html">
- Comet Plugin Overview
+ Comet Plugin Architecture
</a>
</li>
<li class="toctree-l1">
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org