You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/06 14:01:01 UTC
(datafusion) branch asf-site updated: Publish built docs triggered by 6f3f7b7420c58bedf795260ddb3677be1b90b1af
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 58407a125c Publish built docs triggered by 6f3f7b7420c58bedf795260ddb3677be1b90b1af
58407a125c is described below
commit 58407a125ce7f9e134fbe6bddaf7691f6a0e685a
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Tue Oct 6 14:00:54 2026 +0000
Publish built docs triggered by 6f3f7b7420c58bedf795260ddb3677be1b90b1af
---
_sources/library-user-guide/query-optimizer.md.txt | 10 ++++++++++
library-user-guide/query-optimizer.html | 10 ++++++++++
searchindex.js | 2 +-
3 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/_sources/library-user-guide/query-optimizer.md.txt b/_sources/library-user-guide/query-optimizer.md.txt
index a3853d39dd..73c3ed6ed1 100644
--- a/_sources/library-user-guide/query-optimizer.md.txt
+++ b/_sources/library-user-guide/query-optimizer.md.txt
@@ -162,6 +162,16 @@ individual operators or expressions.
Sometimes there is an initial pass that visits the plan and builds state that is used in a second pass that performs
the actual optimization. This approach is used in projection push down and filter push down.
+### Rule Precedence
+
+Two rules can want the opposite order for the same pair of adjacent plan nodes. Each rule then undoes the work of the other one on every optimizer pass. The plan does not reach a fixed point, so the optimizer runs more passes than necessary, and the position of the two rules in the rule list decides the plan. The optimizer can stop early when the plan at the end of a pass is the same as the plan at the end of an earlier pass, but that does not make the result independent of the rule order.
+
+Do not let two rules compete. Give one rule precedence, make the competing rule yield, and record the decision in the module documentation of both rules.
+
+There is one such decision today:
+
+- `PushDownFilter` yields to a _pure extraction projection_. A pure extraction projection is a projection whose expressions are only `__datafusion_extracted_N` aliases and pass-through columns. `ExtractLeafExpressions` creates it, and `PushDownLeafProjections` moves it towards the leaves. `PushDownFilter` does not move a filter below such a projection, so the projection stays next to the scan. A Parquet scan then merges the projection into the file projection and reads only the struct le [...]
+
### Expression Naming
Every expression in DataFusion has a name, which is used as the column name. For example, in this example the output
diff --git a/library-user-guide/query-optimizer.html b/library-user-guide/query-optimizer.html
index 124305b0ac..94f78cfe8a 100644
--- a/library-user-guide/query-optimizer.html
+++ b/library-user-guide/query-optimizer.html
@@ -540,6 +540,15 @@ individual operators or expressions.</p>
<p>Sometimes there is an initial pass that visits the plan and builds state that is used in a second pass that performs
the actual optimization. This approach is used in projection push down and filter push down.</p>
</section>
+<section id="rule-precedence">
+<h3>Rule Precedence<a class="headerlink" href="#rule-precedence" title="Link to this heading">#</a></h3>
+<p>Two rules can want the opposite order for the same pair of adjacent plan nodes. Each rule then undoes the work of the other one on every optimizer pass. The plan does not reach a fixed point, so the optimizer runs more passes than necessary, and the position of the two rules in the rule list decides the plan. The optimizer can stop early when the plan at the end of a pass is the same as the plan at the end of an earlier pass, but that does not make the result independent of the rule o [...]
+<p>Do not let two rules compete. Give one rule precedence, make the competing rule yield, and record the decision in the module documentation of both rules.</p>
+<p>There is one such decision today:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">PushDownFilter</span></code> yields to a <em>pure extraction projection</em>. A pure extraction projection is a projection whose expressions are only <code class="docutils literal notranslate"><span class="pre">__datafusion_extracted_N</span></code> aliases and pass-through columns. <code class="docutils literal notranslate"><span class="pre">ExtractLeafExpressions</span></code> creates it, and <code class="docutils lite [...]
+</ul>
+</section>
<section id="expression-naming">
<h3>Expression Naming<a class="headerlink" href="#expression-naming" title="Link to this heading">#</a></h3>
<p>Every expression in DataFusion has a name, which is used as the column name. For example, in this example the output
@@ -905,6 +914,7 @@ take.</p>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#writing-optimization-rules">Writing Optimization Rules</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#providing-custom-rules">Providing Custom Rules</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#general-guidelines">General Guidelines</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#rule-precedence">Rule Precedence</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#expression-naming">Expression Naming</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#implication">Implication</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#building-expression-names">Building Expression Names</a></li>
diff --git a/searchindex.js b/searchindex.js
index 82bc46c5d8..aa351ddb64 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[77,"op-neq"]],"!~":[[77,"op-re-not-match"]],"!~*":[[77,"op-re-not-match-i"]],"!~~":[[77,"id19"]],"!~~*":[[77,"id20"]],"#":[[77,"op-bit-xor"]],"%":[[77,"op-modulo"]],"&":[[77,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[16,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[77,"op-multiply"]],"+":[[77,"op-plus"]],"-":[[77,"op-minus"]],"/":[[77,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[77,"op-neq"]],"!~":[[77,"op-re-not-match"]],"!~*":[[77,"op-re-not-match-i"]],"!~~":[[77,"id19"]],"!~~*":[[77,"id20"]],"#":[[77,"op-bit-xor"]],"%":[[77,"op-modulo"]],"&":[[77,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[16,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[77,"op-multiply"]],"+":[[77,"op-plus"]],"-":[[77,"op-minus"]],"/":[[77,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org