You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2025/05/09 20:37:56 UTC
(datafusion-comet) branch asf-site updated: Publish built docs triggered by 25e39ab8ba2b882057c5a5c44be79d88015890fb
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 2e4b570d6 Publish built docs triggered by 25e39ab8ba2b882057c5a5c44be79d88015890fb
2e4b570d6 is described below
commit 2e4b570d61e8e9610dd55f2e17315c2e9c56915f
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri May 9 20:37:52 2025 +0000
Publish built docs triggered by 25e39ab8ba2b882057c5a5c44be79d88015890fb
---
_images/tracing.png | Bin 0 -> 95661 bytes
_sources/contributor-guide/tracing.md.txt | 52 ++++++++++
_sources/index.rst.txt | 1 +
_static/images/tracing.png | Bin 0 -> 95661 bytes
contributor-guide/adding_a_new_expression.html | 11 ++-
contributor-guide/benchmark-results/tpc-ds.html | 5 +
contributor-guide/benchmark-results/tpc-h.html | 5 +
contributor-guide/benchmarking.html | 5 +
contributor-guide/benchmarking_aws_ec2.html | 5 +
contributor-guide/benchmarking_macos.html | 5 +
contributor-guide/contributing.html | 5 +
contributor-guide/debugging.html | 5 +
contributor-guide/development.html | 5 +
contributor-guide/plugin_overview.html | 5 +
contributor-guide/profiling_native_code.html | 11 ++-
contributor-guide/spark-sql-tests.html | 5 +
.../{profiling_native_code.html => tracing.html} | 105 ++++++---------------
genindex.html | 5 +
index.html | 6 ++
objects.inv | Bin 872 -> 885 bytes
search.html | 5 +
searchindex.js | 2 +-
user-guide/compatibility.html | 5 +
user-guide/configs.html | 5 +
user-guide/datasources.html | 5 +
user-guide/datatypes.html | 5 +
user-guide/expressions.html | 5 +
user-guide/iceberg.html | 5 +
user-guide/installation.html | 5 +
user-guide/kubernetes.html | 5 +
user-guide/metrics.html | 5 +
user-guide/operators.html | 5 +
user-guide/overview.html | 5 +
user-guide/source.html | 5 +
user-guide/tuning.html | 5 +
35 files changed, 230 insertions(+), 83 deletions(-)
diff --git a/_images/tracing.png b/_images/tracing.png
new file mode 100644
index 000000000..78897512f
Binary files /dev/null and b/_images/tracing.png differ
diff --git a/_sources/contributor-guide/tracing.md.txt b/_sources/contributor-guide/tracing.md.txt
new file mode 100644
index 000000000..47c0cc2f2
--- /dev/null
+++ b/_sources/contributor-guide/tracing.md.txt
@@ -0,0 +1,52 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Tracing
+
+Tracing can be enabled by setting `spark.comet.tracing.enabled=true`. Additionally, enabling the `jemalloc`
+feature will enable tracing of native memory allocations.
+
+```shell
+make release COMET_FEATURES="jemalloc"
+```
+
+With this feature enabled, each Spark executor will write a JSON event log file in
+Chrome's [Trace Event Format].
+
+[Trace Event Format]: https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview?tab=t.0#heading=h.yr4qxyxotyw
+
+Example output:
+
+```json
+{ "name": "decodeShuffleBlock", "cat": "PERF", "ph": "B", "pid": 1, "tid": 5, "ts": 10109225730 },
+{ "name": "decodeShuffleBlock", "cat": "PERF", "ph": "E", "pid": 1, "tid": 5, "ts": 10109228835 },
+{ "name": "decodeShuffleBlock", "cat": "PERF", "ph": "B", "pid": 1, "tid": 5, "ts": 10109245928 },
+{ "name": "decodeShuffleBlock", "cat": "PERF", "ph": "E", "pid": 1, "tid": 5, "ts": 10109248843 },
+{ "name": "execute_plan", "cat": "PERF", "ph": "E", "pid": 1, "tid": 5, "ts": 10109350935 },
+{ "name": "CometExecIterator_getNextBatch", "cat": "PERF", "ph": "E", "pid": 1, "tid": 5, "ts": 10109367116 },
+{ "name": "CometExecIterator_getNextBatch", "cat": "PERF", "ph": "B", "pid": 1, "tid": 5, "ts": 10109479156 },
+```
+
+Traces can be viewed with [Trace Viewer].
+
+[Trace Viewer]: https://github.com/catapult-project/catapult/blob/main/tracing/README.md
+
+Example trace visualization:
+
+
diff --git a/_sources/index.rst.txt b/_sources/index.rst.txt
index 21ec36ca9..e8efc2ea7 100644
--- a/_sources/index.rst.txt
+++ b/_sources/index.rst.txt
@@ -64,6 +64,7 @@ as a native runtime to achieve improvement in terms of query efficiency and quer
Debugging Guide <contributor-guide/debugging>
Benchmarking Guide <contributor-guide/benchmarking>
Adding a New Expression <contributor-guide/adding_a_new_expression>
+ Tracing <contributor-guide/tracing>
Profiling Native Code <contributor-guide/profiling_native_code>
Spark SQL Tests <contributor-guide/spark-sql-tests.md>
Github and Issue Tracker <https://github.com/apache/datafusion-comet>
diff --git a/_static/images/tracing.png b/_static/images/tracing.png
new file mode 100644
index 000000000..78897512f
Binary files /dev/null and b/_static/images/tracing.png differ
diff --git a/contributor-guide/adding_a_new_expression.html b/contributor-guide/adding_a_new_expression.html
index 23f88d096..5a02b1e90 100644
--- a/contributor-guide/adding_a_new_expression.html
+++ b/contributor-guide/adding_a_new_expression.html
@@ -53,7 +53,7 @@ under the License.
<script async="true" defer="true" src="https://buttons.github.io/buttons.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Profiling Native Code" href="profiling_native_code.html" />
+ <link rel="next" title="Tracing" href="tracing.html" />
<link rel="prev" title="Comet Benchmarking Guide" href="benchmarking.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
@@ -590,10 +595,10 @@ under the License.
<p class="prev-next-title">Comet Benchmarking Guide</p>
</div>
</a>
- <a class='right-next' id="next-link" href="profiling_native_code.html" title="next page">
+ <a class='right-next' id="next-link" href="tracing.html" title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Profiling Native Code</p>
+ <p class="prev-next-title">Tracing</p>
</div>
<i class="fas fa-angle-right"></i>
</a>
diff --git a/contributor-guide/benchmark-results/tpc-ds.html b/contributor-guide/benchmark-results/tpc-ds.html
index 806c9254a..6f1e55dd3 100644
--- a/contributor-guide/benchmark-results/tpc-ds.html
+++ b/contributor-guide/benchmark-results/tpc-ds.html
@@ -203,6 +203,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/benchmark-results/tpc-h.html b/contributor-guide/benchmark-results/tpc-h.html
index 8b624c10a..73e6eb003 100644
--- a/contributor-guide/benchmark-results/tpc-h.html
+++ b/contributor-guide/benchmark-results/tpc-h.html
@@ -203,6 +203,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/benchmarking.html b/contributor-guide/benchmarking.html
index 0e244fd2e..fce2c0b29 100644
--- a/contributor-guide/benchmarking.html
+++ b/contributor-guide/benchmarking.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/benchmarking_aws_ec2.html b/contributor-guide/benchmarking_aws_ec2.html
index 90b7ef552..a6b77f548 100644
--- a/contributor-guide/benchmarking_aws_ec2.html
+++ b/contributor-guide/benchmarking_aws_ec2.html
@@ -203,6 +203,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/benchmarking_macos.html b/contributor-guide/benchmarking_macos.html
index e09bc74b6..73ad949d6 100644
--- a/contributor-guide/benchmarking_macos.html
+++ b/contributor-guide/benchmarking_macos.html
@@ -203,6 +203,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/contributing.html b/contributor-guide/contributing.html
index d4848d73e..949c127a4 100644
--- a/contributor-guide/contributing.html
+++ b/contributor-guide/contributing.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/debugging.html b/contributor-guide/debugging.html
index 23ac1073d..de9ae5b37 100644
--- a/contributor-guide/debugging.html
+++ b/contributor-guide/debugging.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/development.html b/contributor-guide/development.html
index 22e29fe7a..47074c70b 100644
--- a/contributor-guide/development.html
+++ b/contributor-guide/development.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/plugin_overview.html b/contributor-guide/plugin_overview.html
index 33ef28173..335c7c3d2 100644
--- a/contributor-guide/plugin_overview.html
+++ b/contributor-guide/plugin_overview.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/profiling_native_code.html b/contributor-guide/profiling_native_code.html
index 014fb9c95..3d71ba0f3 100644
--- a/contributor-guide/profiling_native_code.html
+++ b/contributor-guide/profiling_native_code.html
@@ -54,7 +54,7 @@ under the License.
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Running Spark SQL Tests" href="spark-sql-tests.html" />
- <link rel="prev" title="Adding a New Expression" href="adding_a_new_expression.html" />
+ <link rel="prev" title="Tracing" href="tracing.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1 current active">
<a class="current reference internal" href="#">
Profiling Native Code
@@ -409,11 +414,11 @@ running flamegraph.</p>
<!-- Previous / next buttons -->
<div class='prev-next-area'>
- <a class='left-prev' id="prev-link" href="adding_a_new_expression.html" title="previous page">
+ <a class='left-prev' id="prev-link" href="tracing.html" title="previous page">
<i class="fas fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Adding a New Expression</p>
+ <p class="prev-next-title">Tracing</p>
</div>
</a>
<a class='right-next' id="next-link" href="spark-sql-tests.html" title="next page">
diff --git a/contributor-guide/spark-sql-tests.html b/contributor-guide/spark-sql-tests.html
index e4e65e7b3..7ec3aac82 100644
--- a/contributor-guide/spark-sql-tests.html
+++ b/contributor-guide/spark-sql-tests.html
@@ -204,6 +204,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
diff --git a/contributor-guide/profiling_native_code.html b/contributor-guide/tracing.html
similarity index 55%
copy from contributor-guide/profiling_native_code.html
copy to contributor-guide/tracing.html
index 014fb9c95..52b4c94dd 100644
--- a/contributor-guide/profiling_native_code.html
+++ b/contributor-guide/tracing.html
@@ -24,7 +24,7 @@ under the License.
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
- <title>Profiling Native Code — Apache DataFusion Comet documentation</title>
+ <title>Tracing — Apache DataFusion Comet documentation</title>
<link href="../_static/styles/theme.css?digest=1999514e3f237ded88cf" rel="stylesheet">
<link href="../_static/styles/pydata-sphinx-theme.css?digest=1999514e3f237ded88cf" rel="stylesheet">
@@ -53,7 +53,7 @@ under the License.
<script async="true" defer="true" src="https://buttons.github.io/buttons.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Running Spark SQL Tests" href="spark-sql-tests.html" />
+ <link rel="next" title="Profiling Native Code" href="profiling_native_code.html" />
<link rel="prev" title="Adding a New Expression" href="adding_a_new_expression.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -207,6 +207,11 @@ under the License.
</li>
<li class="toctree-l1 current active">
<a class="current reference internal" href="#">
+ Tracing
+ </a>
+ </li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling_native_code.html">
Profiling Native Code
</a>
</li>
@@ -280,24 +285,9 @@ under the License.
<div class="toc-item">
-<div class="tocsection onthispage pt-5 pb-3">
- <i class="fas fa-list"></i> On this page
-</div>
<nav id="bd-toc-nav">
- <ul class="visible nav section-nav flex-column">
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#running-micro-benchmarks-with-cargo-bench">
- Running micro benchmarks with cargo bench
- </a>
- </li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#profiling-with-cargo-flamegraph">
- Profiling with cargo-flamegraph
- </a>
- </li>
-</ul>
-
+
</nav>
</div>
@@ -305,7 +295,7 @@ under the License.
<div class="tocsection editthispage">
- <a href="https://github.com/apache/datafusion-comet/edit/main/docs/source/contributor-guide/profiling_native_code.md">
+ <a href="https://github.com/apache/datafusion-comet/edit/main/docs/source/contributor-guide/tracing.md">
<i class="fas fa-pencil-alt"></i> Edit this page
</a>
</div>
@@ -342,65 +332,28 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
-<section id="profiling-native-code">
-<h1>Profiling Native Code<a class="headerlink" href="#profiling-native-code" title="Link to this heading">¶</a></h1>
-<p>We use <code class="docutils literal notranslate"><span class="pre">cargo</span> <span class="pre">bench</span></code> to run benchmarks to measure the performance of individual operators and expressions
-and <a class="reference external" href="https://github.com/flamegraph-rs/flamegraph">cargo-flamegraph</a> for profiling.</p>
-<section id="running-micro-benchmarks-with-cargo-bench">
-<h2>Running micro benchmarks with cargo bench<a class="headerlink" href="#running-micro-benchmarks-with-cargo-bench" title="Link to this heading">¶</a></h2>
-<p>When implementing a new operator or expression, it is good practice to add a new microbenchmark under <code class="docutils literal notranslate"><span class="pre">core/benches</span></code>.</p>
-<p>It is often easiest to copy an existing benchmark and modify it for the new operator or expression. It is also
-necessary to add a new section to the <code class="docutils literal notranslate"><span class="pre">Cargo.toml</span></code> file, such as:</p>
-<div class="highlight-toml notranslate"><div class="highlight"><pre><span></span><span class="k">[[bench]]</span>
-<span class="n">name</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"shuffle_writer"</span>
-<span class="n">harness</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">false</span>
+<section id="tracing">
+<h1>Tracing<a class="headerlink" href="#tracing" title="Link to this heading">¶</a></h1>
+<p>Tracing can be enabled by setting <code class="docutils literal notranslate"><span class="pre">spark.comet.tracing.enabled=true</span></code>. Additionally, enabling the <code class="docutils literal notranslate"><span class="pre">jemalloc</span></code>
+feature will enable tracing of native memory allocations.</p>
+<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>make<span class="w"> </span>release<span class="w"> </span><span class="nv">COMET_FEATURES</span><span class="o">=</span><span class="s2">"jemalloc"</span>
</pre></div>
</div>
-<p>These benchmarks are useful when for comparing performance between releases or between feature branches and the
-main branch to help prevent regressions in performance when adding new features or fixing bugs.</p>
-<p>Individual benchmarks can be run by name with the following command.</p>
-<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>cargo<span class="w"> </span>bench<span class="w"> </span>shuffle_writer
+<p>With this feature enabled, each Spark executor will write a JSON event log file in
+Chrome’s <a class="reference external" href="https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview?tab=t.0#heading=h.yr4qxyxotyw">Trace Event Format</a>.</p>
+<p>Example output:</p>
+<div class="highlight-json notranslate"><div class="highlight"><pre><span></span><span class="p">{</span><span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"decodeShuffleBlock"</span><span class="p">,</span><span class="w"> </span><span class="nt">"cat"</span><span class="p">:</span><span class="w"> </span><span class="s2">"PERF"</span><span class="p">,</span><span class="w"> </spa [...]
+<span class="p">{</span><span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"decodeShuffleBlock"</span><span class="p">,</span><span class="w"> </span><span class="nt">"cat"</span><span class="p">:</span><span class="w"> </span><span class="s2">"PERF"</span><span class="p">,</span><span class="w"> </span><span class="nt">"ph"</span><span class="p">:</span><span class="w"> [...]
+<span class="p">{</span><span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"decodeShuffleBlock"</span><span class="p">,</span><span class="w"> </span><span class="nt">"cat"</span><span class="p">:</span><span class="w"> </span><span class="s2">"PERF"</span><span class="p">,</span><span class="w"> </span><span class="nt">"ph"</span><span class="p">:</span><span class="w"> [...]
+<span class="p">{</span><span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"decodeShuffleBlock"</span><span class="p">,</span><span class="w"> </span><span class="nt">"cat"</span><span class="p">:</span><span class="w"> </span><span class="s2">"PERF"</span><span class="p">,</span><span class="w"> </span><span class="nt">"ph"</span><span class="p">:</span><span class="w"> [...]
+<span class="p">{</span><span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"execute_plan"</span><span class="p">,</span><span class="w"> </span><span class="nt">"cat"</span><span class="p">:</span><span class="w"> </span><span class="s2">"PERF"</span><span class="p">,</span><span class="w"> </span><span class="nt">"ph"</span><span class="p">:</span><span class="w"> </span [...]
+<span class="p">{</span><span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"CometExecIterator_getNextBatch"</span><span class="p">,</span><span class="w"> </span><span class="nt">"cat"</span><span class="p">:</span><span class="w"> </span><span class="s2">"PERF"</span><span class="p">,</span><span class="w"> </span><span class="nt">"ph"</span><span class="p">:</span><span [...]
+<span class="p">{</span><span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"CometExecIterator_getNextBatch"</span><span class="p">,</span><span class="w"> </span><span class="nt">"cat"</span><span class="p">:</span><span class="w"> </span><span class="s2">"PERF"</span><span class="p">,</span><span class="w"> </span><span class="nt">"ph"</span><span class="p">:</span><span [...]
</pre></div>
</div>
-<p>Here is some sample output from running this command.</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span></span> <span class="n">Running</span> <span class="n">benches</span><span class="o">/</span><span class="n">shuffle_writer</span><span class="o">.</span><span class="n">rs</span> <span class="p">(</span><span class="n">target</span><span class="o">/</span><span class="n">release</span><span class="o">/</span><span class="n">deps</span><span class="o">/</span><span class="n">shuffle_writer</span><span class= [...]
-<span class="n">Gnuplot</span> <span class="ow">not</span> <span class="n">found</span><span class="p">,</span> <span class="n">using</span> <span class="n">plotters</span> <span class="n">backend</span>
-<span class="n">shuffle_writer</span><span class="o">/</span><span class="n">shuffle_writer</span>
- <span class="n">time</span><span class="p">:</span> <span class="p">[</span><span class="mf">2.0880</span> <span class="n">ms</span> <span class="mf">2.0989</span> <span class="n">ms</span> <span class="mf">2.1118</span> <span class="n">ms</span><span class="p">]</span>
-<span class="n">Found</span> <span class="mi">9</span> <span class="n">outliers</span> <span class="n">among</span> <span class="mi">100</span> <span class="n">measurements</span> <span class="p">(</span><span class="mf">9.00</span><span class="o">%</span><span class="p">)</span>
- <span class="mi">3</span> <span class="p">(</span><span class="mf">3.00</span><span class="o">%</span><span class="p">)</span> <span class="n">high</span> <span class="n">mild</span>
- <span class="mi">6</span> <span class="p">(</span><span class="mf">6.00</span><span class="o">%</span><span class="p">)</span> <span class="n">high</span> <span class="n">severe</span>
-</pre></div>
-</div>
-</section>
-<section id="profiling-with-cargo-flamegraph">
-<h2>Profiling with cargo-flamegraph<a class="headerlink" href="#profiling-with-cargo-flamegraph" title="Link to this heading">¶</a></h2>
-<p>Install cargo-flamegraph:</p>
-<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>cargo<span class="w"> </span>install<span class="w"> </span>flamegraph
-</pre></div>
-</div>
-<p>Follow the instructions in <a class="reference external" href="https://github.com/flamegraph-rs/flamegraph">cargo-flamegraph</a> for your platform for
-running flamegraph.</p>
-<p>Here is a sample command for running <code class="docutils literal notranslate"><span class="pre">cargo-flamegraph</span></code> on MacOS.</p>
-<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>cargo<span class="w"> </span>flamegraph<span class="w"> </span>--root<span class="w"> </span>--bench<span class="w"> </span>shuffle_writer
-</pre></div>
-</div>
-<p>This will produce output similar to the following.</p>
-<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dtrace</span><span class="p">:</span> <span class="n">system</span> <span class="n">integrity</span> <span class="n">protection</span> <span class="ow">is</span> <span class="n">on</span><span class="p">,</span> <span class="n">some</span> <span class="n">features</span> <span class="n">will</span> <span class="ow">not</span> <span class="n">be</span> <span class="n">available</span>
-
-<span class="n">dtrace</span><span class="p">:</span> <span class="n">description</span> <span class="s1">'profile-997 '</span> <span class="n">matched</span> <span class="mi">1</span> <span class="n">probe</span>
-<span class="n">Gnuplot</span> <span class="ow">not</span> <span class="n">found</span><span class="p">,</span> <span class="n">using</span> <span class="n">plotters</span> <span class="n">backend</span>
-<span class="n">Testing</span> <span class="n">shuffle_writer</span><span class="o">/</span><span class="n">shuffle_writer</span>
-<span class="n">Success</span>
-
-<span class="n">dtrace</span><span class="p">:</span> <span class="n">pid</span> <span class="mi">66402</span> <span class="n">has</span> <span class="n">exited</span>
-<span class="n">writing</span> <span class="n">flamegraph</span> <span class="n">to</span> <span class="s2">"flamegraph.svg"</span>
-</pre></div>
-</div>
-<p>The generated flamegraph can now be opened in a browser that supports svg format.</p>
-<p>Here is the flamegraph for this example:</p>
-<p><img alt="flamegraph" src="../_images/flamegraph.png" /></p>
-</section>
+<p>Traces can be viewed with <a class="reference external" href="https://github.com/catapult-project/catapult/blob/main/tracing/README.md">Trace Viewer</a>.</p>
+<p>Example trace visualization:</p>
+<p><img alt="tracing" src="../_images/tracing.png" /></p>
</section>
@@ -416,10 +369,10 @@ running flamegraph.</p>
<p class="prev-next-title">Adding a New Expression</p>
</div>
</a>
- <a class='right-next' id="next-link" href="spark-sql-tests.html" title="next page">
+ <a class='right-next' id="next-link" href="profiling_native_code.html" title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Running Spark SQL Tests</p>
+ <p class="prev-next-title">Profiling Native Code</p>
</div>
<i class="fas fa-angle-right"></i>
</a>
diff --git a/genindex.html b/genindex.html
index 4d85af0cb..b3fd33c5a 100644
--- a/genindex.html
+++ b/genindex.html
@@ -202,6 +202,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/index.html b/index.html
index c69eaf3b1..af2348b85 100644
--- a/index.html
+++ b/index.html
@@ -204,6 +204,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="contributor-guide/profiling_native_code.html">
Profiling Native Code
@@ -346,6 +351,7 @@ as a native runtime to achieve improvement in terms of query efficiency and quer
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/debugging.html">Debugging Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/benchmarking.html">Benchmarking Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/adding_a_new_expression.html">Adding a New Expression</a></li>
+<li class="toctree-l1"><a class="reference internal" href="contributor-guide/tracing.html">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/profiling_native_code.html">Profiling Native Code</a></li>
<li class="toctree-l1"><a class="reference internal" href="contributor-guide/spark-sql-tests.html">Spark SQL Tests</a></li>
<li class="toctree-l1"><a class="reference external" href="https://github.com/apache/datafusion-comet">Github and Issue Tracker</a></li>
diff --git a/objects.inv b/objects.inv
index 0bb0be421..3aa1a6dc5 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/search.html b/search.html
index fb7a5b03e..0e665d816 100644
--- a/search.html
+++ b/search.html
@@ -209,6 +209,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/searchindex.js b/searchindex.js
index 0042e9590..bf76d901e 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Install Comet": [[11, "install-comet"]], "2. Clone Spark and Apply Diff": [[11, "clone-spark-and-apply-diff"]], "3. Run Spark SQL Tests": [[11, "run-spark-sql-tests"]], "ANSI mode": [[13, "ansi-mode"]], "API Differences Between Spark Versions": [[0, "api-differences-between-spark-versions"]], "ASF Links": [[12, null]], "Accelerating Apache Iceberg Parquet Scans using Comet (Experimental)": [[18, null]], "Adding Spark-side Tests for the New Expression": [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Install Comet": [[11, "install-comet"]], "2. Clone Spark and Apply Diff": [[11, "clone-spark-and-apply-diff"]], "3. Run Spark SQL Tests": [[11, "run-spark-sql-tests"]], "ANSI mode": [[14, "ansi-mode"]], "API Differences Between Spark Versions": [[0, "api-differences-between-spark-versions"]], "ASF Links": [[13, null]], "Accelerating Apache Iceberg Parquet Scans using Comet (Experimental)": [[19, null]], "Adding Spark-side Tests for the New Expression": [...]
\ No newline at end of file
diff --git a/user-guide/compatibility.html b/user-guide/compatibility.html
index 950e0782c..9be020c93 100644
--- a/user-guide/compatibility.html
+++ b/user-guide/compatibility.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/configs.html b/user-guide/configs.html
index cf1e1941c..43e518183 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/datasources.html b/user-guide/datasources.html
index 4d1a7f70e..772109d4c 100644
--- a/user-guide/datasources.html
+++ b/user-guide/datasources.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/datatypes.html b/user-guide/datatypes.html
index 84ccb47ed..28529b363 100644
--- a/user-guide/datatypes.html
+++ b/user-guide/datatypes.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/expressions.html b/user-guide/expressions.html
index f6c0d27c0..4d6072903 100644
--- a/user-guide/expressions.html
+++ b/user-guide/expressions.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/iceberg.html b/user-guide/iceberg.html
index ab55b89dc..002b33afb 100644
--- a/user-guide/iceberg.html
+++ b/user-guide/iceberg.html
@@ -203,6 +203,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/installation.html b/user-guide/installation.html
index 5a4a859c6..e9d2ab870 100644
--- a/user-guide/installation.html
+++ b/user-guide/installation.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/kubernetes.html b/user-guide/kubernetes.html
index d4e131c3e..56748ff2b 100644
--- a/user-guide/kubernetes.html
+++ b/user-guide/kubernetes.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/metrics.html b/user-guide/metrics.html
index e685f13e9..7e1b0110f 100644
--- a/user-guide/metrics.html
+++ b/user-guide/metrics.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/operators.html b/user-guide/operators.html
index 7b5b60092..b9855221e 100644
--- a/user-guide/operators.html
+++ b/user-guide/operators.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/overview.html b/user-guide/overview.html
index 7b273bf5f..e61f5ee32 100644
--- a/user-guide/overview.html
+++ b/user-guide/overview.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/source.html b/user-guide/source.html
index b5a151941..08ef6f7c2 100644
--- a/user-guide/source.html
+++ b/user-guide/source.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
diff --git a/user-guide/tuning.html b/user-guide/tuning.html
index eb07a80e5..b1174385b 100644
--- a/user-guide/tuning.html
+++ b/user-guide/tuning.html
@@ -205,6 +205,11 @@ under the License.
Adding a New Expression
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../contributor-guide/tracing.html">
+ Tracing
+ </a>
+ </li>
<li class="toctree-l1">
<a class="reference internal" href="../contributor-guide/profiling_native_code.html">
Profiling Native Code
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org