You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/06/02 22:21:53 UTC
(datafusion) branch asf-site updated: Publish built docs triggered by 488a5845223b0be9520ef6eb6060aa8c490f2003
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 445d807065 Publish built docs triggered by 488a5845223b0be9520ef6eb6060aa8c490f2003
445d807065 is described below
commit 445d807065b6755e75d5c58af03cc312e0a38676
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Tue Jun 2 22:21:47 2026 +0000
Publish built docs triggered by 488a5845223b0be9520ef6eb6060aa8c490f2003
---
_sources/contributor-guide/testing.md.txt | 12 ++++++++++++
contributor-guide/testing.html | 17 ++++++++++++++++-
searchindex.js | 2 +-
3 files changed, 29 insertions(+), 2 deletions(-)
diff --git a/_sources/contributor-guide/testing.md.txt b/_sources/contributor-guide/testing.md.txt
index 3b644f610b..3e44e3aaba 100644
--- a/_sources/contributor-guide/testing.md.txt
+++ b/_sources/contributor-guide/testing.md.txt
@@ -113,6 +113,18 @@ Like similar systems such as [DuckDB](https://duckdb.org/dev/testing), DataFusio
DataFusion has integrated [sqlite's test suite](https://sqlite.org/sqllogictest/doc/trunk/about.wiki) as a supplemental test suite that is run whenever a PR is merged into DataFusion. To run it manually please refer to the [README](https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-sqlite) file for instructions.
+### Allocator-level memory accounting (`--features memory-accounting`)
+
+For tests that need to verify DataFusion's voluntary memory tracking
+matches actual heap usage, the `sqllogictest` runner ships an optional
+`memory-accounting` feature that installs a global allocator wrapper.
+Adding `SET datafusion.runtime.memory_limit = 'N'` at the top of an
+`.slt` file opts that file into allocator-vs-`MemoryPool` reconciliation
+with 10% headroom — any divergence panics the test with an
+`OverdraftPanic` reporting the actual allocator balance. See
+[the sqllogictest README](https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-allocator-level-memory-accounting)
+for the runner flag and the full mechanism.
+
## Snapshot testing (`cargo insta`)
[Insta](https://github.com/mitsuhiko/insta) is used for snapshot testing. Snapshots are generated
diff --git a/contributor-guide/testing.html b/contributor-guide/testing.html
index 7e2b7463b0..a2cda32cde 100644
--- a/contributor-guide/testing.html
+++ b/contributor-guide/testing.html
@@ -506,6 +506,18 @@ cargo<span class="w"> </span><span class="nb">test</span><span class="w"> </span
<p><code class="docutils literal notranslate"><span class="pre">sqllogictests</span></code> may be less convenient for new contributors who are familiar with writing <code class="docutils literal notranslate"><span class="pre">.rs</span></code> tests as they require learning another tool. However, <code class="docutils literal notranslate"><span class="pre">sqllogictest</span></code> based tests are much easier to develop and maintain as they 1) do not require a slow recompile/link cycle [...]
<p>Like similar systems such as <a class="reference external" href="https://duckdb.org/dev/testing">DuckDB</a>, DataFusion has chosen to trade off a slightly higher barrier to contribution for longer term maintainability.</p>
<p>DataFusion has integrated <a class="reference external" href="https://sqlite.org/sqllogictest/doc/trunk/about.wiki">sqlite’s test suite</a> as a supplemental test suite that is run whenever a PR is merged into DataFusion. To run it manually please refer to the <a class="reference external" href="https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-sqlite">README</a> file for instructions.</p>
+<section id="allocator-level-memory-accounting-features-memory-accounting">
+<h3>Allocator-level memory accounting (<code class="docutils literal notranslate"><span class="pre">--features</span> <span class="pre">memory-accounting</span></code>)<a class="headerlink" href="#allocator-level-memory-accounting-features-memory-accounting" title="Link to this heading">#</a></h3>
+<p>For tests that need to verify DataFusion’s voluntary memory tracking
+matches actual heap usage, the <code class="docutils literal notranslate"><span class="pre">sqllogictest</span></code> runner ships an optional
+<code class="docutils literal notranslate"><span class="pre">memory-accounting</span></code> feature that installs a global allocator wrapper.
+Adding <code class="docutils literal notranslate"><span class="pre">SET</span> <span class="pre">datafusion.runtime.memory_limit</span> <span class="pre">=</span> <span class="pre">'N'</span></code> at the top of an
+<code class="docutils literal notranslate"><span class="pre">.slt</span></code> file opts that file into allocator-vs-<code class="docutils literal notranslate"><span class="pre">MemoryPool</span></code> reconciliation
+with 10% headroom — any divergence panics the test with an
+<code class="docutils literal notranslate"><span class="pre">OverdraftPanic</span></code> reporting the actual allocator balance. See
+<a class="reference external" href="https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-allocator-level-memory-accounting">the sqllogictest README</a>
+for the runner flag and the full mechanism.</p>
+</section>
</section>
<section id="snapshot-testing-cargo-insta">
<h2>Snapshot testing (<code class="docutils literal notranslate"><span class="pre">cargo</span> <span class="pre">insta</span></code>)<a class="headerlink" href="#snapshot-testing-cargo-insta" title="Link to this heading">#</a></h2>
@@ -680,7 +692,10 @@ bash<span class="w"> </span>ci/scripts/markdown_link_check.sh
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#testing-quick-start">Testing Quick Start</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#testing-overview">Testing Overview</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#unit-tests">Unit tests</a></li>
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#sqllogictests-tests">sqllogictests Tests</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#sqllogictests-tests">sqllogictests Tests</a><ul class="nav section-nav flex-column">
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#allocator-level-memory-accounting-features-memory-accounting">Allocator-level memory accounting (<code class="docutils literal notranslate"><span class="pre">--features</span> <span class="pre">memory-accounting</span></code>)</a></li>
+</ul>
+</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#snapshot-testing-cargo-insta">Snapshot testing (<code class="docutils literal notranslate"><span class="pre">cargo</span> <span class="pre">insta</span></code>)</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#extended-tests">Extended Tests</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#rust-integration-tests">Rust Integration Tests</a></li>
diff --git a/searchindex.js b/searchindex.js
index e5c95b6d97..f9c96ce718 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[74,"op-neq"]],"!~":[[74,"op-re-not-match"]],"!~*":[[74,"op-re-not-match-i"]],"!~~":[[74,"id19"]],"!~~*":[[74,"id20"]],"#":[[74,"op-bit-xor"]],"%":[[74,"op-modulo"]],"&":[[74,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[15,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[74,"op-multiply"]],"+":[[74,"op-plus"]],"-":[[74,"op-minus"]],"/":[[74,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[74,"op-neq"]],"!~":[[74,"op-re-not-match"]],"!~*":[[74,"op-re-not-match-i"]],"!~~":[[74,"id19"]],"!~~*":[[74,"id20"]],"#":[[74,"op-bit-xor"]],"%":[[74,"op-modulo"]],"&":[[74,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[15,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[74,"op-multiply"]],"+":[[74,"op-plus"]],"-":[[74,"op-minus"]],"/":[[74,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org