You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2025/03/21 19:06:44 UTC
(datafusion) branch asf-site updated: Publish built docs triggered by 8756197cf2f1e5994710700129aaa547a6276815
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 9e659b126c Publish built docs triggered by 8756197cf2f1e5994710700129aaa547a6276815
9e659b126c is described below
commit 9e659b126cf05bc170fab0d109e749f3fdafc983
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Mar 21 19:06:40 2025 +0000
Publish built docs triggered by 8756197cf2f1e5994710700129aaa547a6276815
---
.../library-user-guide/working-with-exprs.md.txt | 19 ++++++++++
library-user-guide/working-with-exprs.html | 41 ++++++++++++++++++++++
searchindex.js | 2 +-
3 files changed, 61 insertions(+), 1 deletion(-)
diff --git a/_sources/library-user-guide/working-with-exprs.md.txt b/_sources/library-user-guide/working-with-exprs.md.txt
index d9a6e28a1b..058f4132d4 100644
--- a/_sources/library-user-guide/working-with-exprs.md.txt
+++ b/_sources/library-user-guide/working-with-exprs.md.txt
@@ -50,6 +50,25 @@ As another example, the SQL expression `a + b * c` would be represented as an `E
As the writer of a library, you can use `Expr`s to represent computations that you want to perform. This guide will walk you through how to make your own scalar UDF as an `Expr` and how to rewrite `Expr`s to inline the simple UDF.
+## Arrow Schema and DataFusion DFSchema
+
+Apache Arrow `Schema` provides a lightweight structure for defining data, and Apache Datafusion`DFSchema` extends it with extra information such as column qualifiers and functional dependencies. Column qualifiers are multi part path to the table e.g table, schema, catalog. Functional Dependency is the relationship between attributes(characteristics) of a table related to each other.
+
+### Difference between Schema and DFSchema
+
+- Schema: A fundamental component of Apache Arrow, `Schema` defines a dataset's structure, specifying column names and their data types.
+
+ > Please see [Struct Schema](https://docs.rs/arrow-schema/54.2.1/arrow_schema/struct.Schema.html) for a detailed document of Arrow Schema.
+
+- DFSchema: Extending `Schema`, `DFSchema` incorporates qualifiers such as table names, enabling it to carry additional context when required. This is particularly valuable for managing queries across multiple tables.
+ > Please see [Struct DFSchema](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html) for a detailed document of DFSchema.
+
+### How to convert between Schema and DFSchema
+
+From Schema to DFSchema: Use `DFSchema::try_from_qualified_schema` with a table name and original schema, for detailed code example please see [creating-qualified-schemas](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#creating-qualified-schemas).
+
+From DFSchema to Schema: Since the `Into` trait has been implemented for DFSchema to convert it into an Arrow Schema, for detailed code example please see [converting-back-to-arrow-schema](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#converting-back-to-arrow-schema).
+
## Creating and Evaluating `Expr`s
Please see [expr_api.rs](https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/expr_api.rs) for well commented code for creating, evaluating, simplifying, and analyzing `Expr`s.
diff --git a/library-user-guide/working-with-exprs.html b/library-user-guide/working-with-exprs.html
index 44ae2dda07..6053e6e924 100644
--- a/library-user-guide/working-with-exprs.html
+++ b/library-user-guide/working-with-exprs.html
@@ -522,6 +522,23 @@
<nav id="bd-toc-nav">
<ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#arrow-schema-and-datafusion-dfschema">
+ Arrow Schema and DataFusion DFSchema
+ </a>
+ <ul class="nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#difference-between-schema-and-dfschema">
+ Difference between Schema and DFSchema
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#how-to-convert-between-schema-and-dfschema">
+ How to convert between Schema and DFSchema
+ </a>
+ </li>
+ </ul>
+ </li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#creating-and-evaluating-exprs">
Creating and Evaluating
@@ -659,6 +676,30 @@
</pre></div>
</div>
<p>As the writer of a library, you can use <code class="docutils literal notranslate"><span class="pre">Expr</span></code>s to represent computations that you want to perform. This guide will walk you through how to make your own scalar UDF as an <code class="docutils literal notranslate"><span class="pre">Expr</span></code> and how to rewrite <code class="docutils literal notranslate"><span class="pre">Expr</span></code>s to inline the simple UDF.</p>
+<section id="arrow-schema-and-datafusion-dfschema">
+<h2>Arrow Schema and DataFusion DFSchema<a class="headerlink" href="#arrow-schema-and-datafusion-dfschema" title="Link to this heading">¶</a></h2>
+<p>Apache Arrow <code class="docutils literal notranslate"><span class="pre">Schema</span></code> provides a lightweight structure for defining data, and Apache Datafusion<code class="docutils literal notranslate"><span class="pre">DFSchema</span></code> extends it with extra information such as column qualifiers and functional dependencies. Column qualifiers are multi part path to the table e.g table, schema, catalog. Functional Dependency is the relationship between attributes(characte [...]
+<section id="difference-between-schema-and-dfschema">
+<h3>Difference between Schema and DFSchema<a class="headerlink" href="#difference-between-schema-and-dfschema" title="Link to this heading">¶</a></h3>
+<ul>
+<li><p>Schema: A fundamental component of Apache Arrow, <code class="docutils literal notranslate"><span class="pre">Schema</span></code> defines a dataset’s structure, specifying column names and their data types.</p>
+<blockquote>
+<div><p>Please see <a class="reference external" href="https://docs.rs/arrow-schema/54.2.1/arrow_schema/struct.Schema.html">Struct Schema</a> for a detailed document of Arrow Schema.</p>
+</div></blockquote>
+</li>
+<li><p>DFSchema: Extending <code class="docutils literal notranslate"><span class="pre">Schema</span></code>, <code class="docutils literal notranslate"><span class="pre">DFSchema</span></code> incorporates qualifiers such as table names, enabling it to carry additional context when required. This is particularly valuable for managing queries across multiple tables.</p>
+<blockquote>
+<div><p>Please see <a class="reference external" href="https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html">Struct DFSchema</a> for a detailed document of DFSchema.</p>
+</div></blockquote>
+</li>
+</ul>
+</section>
+<section id="how-to-convert-between-schema-and-dfschema">
+<h3>How to convert between Schema and DFSchema<a class="headerlink" href="#how-to-convert-between-schema-and-dfschema" title="Link to this heading">¶</a></h3>
+<p>From Schema to DFSchema: Use <code class="docutils literal notranslate"><span class="pre">DFSchema::try_from_qualified_schema</span></code> with a table name and original schema, for detailed code example please see <a class="reference external" href="https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#creating-qualified-schemas">creating-qualified-schemas</a>.</p>
+<p>From DFSchema to Schema: Since the <code class="docutils literal notranslate"><span class="pre">Into</span></code> trait has been implemented for DFSchema to convert it into an Arrow Schema, for detailed code example please see <a class="reference external" href="https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#converting-back-to-arrow-schema">converting-back-to-arrow-schema</a>.</p>
+</section>
+</section>
<section id="creating-and-evaluating-exprs">
<h2>Creating and Evaluating <code class="docutils literal notranslate"><span class="pre">Expr</span></code>s<a class="headerlink" href="#creating-and-evaluating-exprs" title="Link to this heading">¶</a></h2>
<p>Please see <a class="reference external" href="https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/expr_api.rs">expr_api.rs</a> for well commented code for creating, evaluating, simplifying, and analyzing <code class="docutils literal notranslate"><span class="pre">Expr</span></code>s.</p>
diff --git a/searchindex.js b/searchindex.js
index 8c9aa8bf49..ef2eb49904 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[52,"op-neq"]],"!~":[[52,"op-re-not-match"]],"!~*":[[52,"op-re-not-match-i"]],"!~~":[[52,"id19"]],"!~~*":[[52,"id20"]],"#":[[52,"op-bit-xor"]],"%":[[52,"op-modulo"]],"&":[[52,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[12,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[52,"op-multiply"]],"+":[[52,"op-plus"]],"-":[[52,"op-minus"]],"/":[[52,"op-divide"]],"2022 Q2":[[10,"q2"]] [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[52,"op-neq"]],"!~":[[52,"op-re-not-match"]],"!~*":[[52,"op-re-not-match-i"]],"!~~":[[52,"id19"]],"!~~*":[[52,"id20"]],"#":[[52,"op-bit-xor"]],"%":[[52,"op-modulo"]],"&":[[52,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[12,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[52,"op-multiply"]],"+":[[52,"op-plus"]],"-":[[52,"op-minus"]],"/":[[52,"op-divide"]],"2022 Q2":[[10,"q2"]] [...]
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org