You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by gi...@apache.org on 2024/01/26 21:57:41 UTC
(arrow-datafusion) branch asf-site updated: Publish built docs triggered by ed2453901949101556c0c89b9acf442873c06ce8
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new a1ca9d407d Publish built docs triggered by ed2453901949101556c0c89b9acf442873c06ce8
a1ca9d407d is described below
commit a1ca9d407da4c38da6287332210667d6fa5ff336
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Fri Jan 26 21:57:35 2024 +0000
Publish built docs triggered by ed2453901949101556c0c89b9acf442873c06ce8
---
_sources/library-user-guide/catalogs.md.txt | 16 ++++++++--------
library-user-guide/catalogs.html | 22 +++++++++++-----------
searchindex.js | 2 +-
3 files changed, 20 insertions(+), 20 deletions(-)
diff --git a/_sources/library-user-guide/catalogs.md.txt b/_sources/library-user-guide/catalogs.md.txt
index e53d163663..d30e26f196 100644
--- a/_sources/library-user-guide/catalogs.md.txt
+++ b/_sources/library-user-guide/catalogs.md.txt
@@ -23,7 +23,7 @@ This section describes how to create and manage catalogs, schemas, and tables in
## General Concepts
-CatalogList, Catalogs, schemas, and tables are organized in a hierarchy. A CatalogList contains catalogs, a catalog contains schemas and a schema contains tables.
+CatalogProviderList, Catalogs, schemas, and tables are organized in a hierarchy. A CatalogProviderList contains catalog providers, a catalog provider contains schemas and a schema contains tables.
DataFusion comes with a basic in memory catalog functionality in the [`catalog` module]. You can use these in memory implementations as is, or extend DataFusion with your own catalog implementations, for example based on local files or files on remote object storage.
@@ -31,9 +31,9 @@ DataFusion comes with a basic in memory catalog functionality in the [`catalog`
Similarly to other concepts in DataFusion, you'll implement various traits to create your own catalogs, schemas, and tables. The following sections describe the traits you'll need to implement.
-The `CatalogList` trait has methods to register new catalogs, get a catalog by name and list all catalogs .The `CatalogProvider` trait has methods to set a schema to a name, get a schema by name, and list all schemas. The `SchemaProvider`, which can be registered with a `CatalogProvider`, has methods to set a table to a name, get a table by name, list all tables, deregister a table, and check for a table's existence. The `TableProvider` trait has methods to scan underlying data and use i [...]
+The `CatalogProviderList` trait has methods to register new catalogs, get a catalog by name and list all catalogs .The `CatalogProvider` trait has methods to set a schema to a name, get a schema by name, and list all schemas. The `SchemaProvider`, which can be registered with a `CatalogProvider`, has methods to set a table to a name, get a table by name, list all tables, deregister a table, and check for a table's existence. The `TableProvider` trait has methods to scan underlying data a [...]
-In the following example, we'll implement an in memory catalog, starting with the `SchemaProvider` trait as we need one to register with the `CatalogProvider`. Finally we will implement `CatalogList` to register the `CatalogProvider`.
+In the following example, we'll implement an in memory catalog, starting with the `SchemaProvider` trait as we need one to register with the `CatalogProvider`. Finally we will implement `CatalogProviderList` to register the `CatalogProvider`.
## Implementing `MemorySchemaProvider`
@@ -169,19 +169,19 @@ impl CatalogProvider for MemoryCatalogProvider {
Again, this is fairly straightforward, as there's an underlying data structure to store the state, via key-value pairs.
-## Implementing `MemoryCatalogList`
+## Implementing `MemoryCatalogProviderList`
```rust
-pub struct MemoryCatalogList {
+pub struct MemoryCatalogProviderList {
/// Collection of catalogs containing schemas and ultimately TableProviders
pub catalogs: DashMap<String, Arc<dyn CatalogProvider>>,
}
```
-With that the `CatalogList` trait can be implemented.
+With that the `CatalogProviderList` trait can be implemented.
```rust
-impl CatalogList for MemoryCatalogList {
+impl CatalogProviderList for MemoryCatalogProviderList {
fn as_any(&self) -> &dyn Any {
self
}
@@ -213,4 +213,4 @@ To recap, you need to:
1. Implement the `TableProvider` trait to create a table provider, or use an existing one.
2. Implement the `SchemaProvider` trait to create a schema provider, or use an existing one.
3. Implement the `CatalogProvider` trait to create a catalog provider, or use an existing one.
-4. Implement the `CatalogList` trait to create a CatalogList, or use an existing one.
+4. Implement the `CatalogProviderList` trait to create a CatalogProviderList, or use an existing one.
diff --git a/library-user-guide/catalogs.html b/library-user-guide/catalogs.html
index ee37931923..ddce04a715 100644
--- a/library-user-guide/catalogs.html
+++ b/library-user-guide/catalogs.html
@@ -390,11 +390,11 @@
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#implementing-memorycataloglist">
+ <a class="reference internal nav-link" href="#implementing-memorycatalogproviderlist">
Implementing
<code class="docutils literal notranslate">
<span class="pre">
- MemoryCatalogList
+ MemoryCatalogProviderList
</span>
</code>
</a>
@@ -455,11 +455,11 @@
<p>This section describes how to create and manage catalogs, schemas, and tables in DataFusion. For those wanting to dive into the code quickly please see the <a class="reference external" href="https://github.com/apache/arrow-datafusion/blob/main/datafusion-examples/examples/catalog.rs">example</a>.</p>
<section id="general-concepts">
<h2>General Concepts<a class="headerlink" href="#general-concepts" title="Link to this heading">¶</a></h2>
-<p>CatalogList, Catalogs, schemas, and tables are organized in a hierarchy. A CatalogList contains catalogs, a catalog contains schemas and a schema contains tables.</p>
+<p>CatalogProviderList, Catalogs, schemas, and tables are organized in a hierarchy. A CatalogProviderList contains catalog providers, a catalog provider contains schemas and a schema contains tables.</p>
<p>DataFusion comes with a basic in memory catalog functionality in the <a class="reference external" href="https://docs.rs/datafusion/latest/datafusion/catalog/index.html"><code class="docutils literal notranslate"><span class="pre">catalog</span></code> module</a>. You can use these in memory implementations as is, or extend DataFusion with your own catalog implementations, for example based on local files or files on remote object storage.</p>
<p>Similarly to other concepts in DataFusion, you’ll implement various traits to create your own catalogs, schemas, and tables. The following sections describe the traits you’ll need to implement.</p>
-<p>The <code class="docutils literal notranslate"><span class="pre">CatalogList</span></code> trait has methods to register new catalogs, get a catalog by name and list all catalogs .The <code class="docutils literal notranslate"><span class="pre">CatalogProvider</span></code> trait has methods to set a schema to a name, get a schema by name, and list all schemas. The <code class="docutils literal notranslate"><span class="pre">SchemaProvider</span></code>, which can be registered with a [...]
-<p>In the following example, we’ll implement an in memory catalog, starting with the <code class="docutils literal notranslate"><span class="pre">SchemaProvider</span></code> trait as we need one to register with the <code class="docutils literal notranslate"><span class="pre">CatalogProvider</span></code>. Finally we will implement <code class="docutils literal notranslate"><span class="pre">CatalogList</span></code> to register the <code class="docutils literal notranslate"><span class [...]
+<p>The <code class="docutils literal notranslate"><span class="pre">CatalogProviderList</span></code> trait has methods to register new catalogs, get a catalog by name and list all catalogs .The <code class="docutils literal notranslate"><span class="pre">CatalogProvider</span></code> trait has methods to set a schema to a name, get a schema by name, and list all schemas. The <code class="docutils literal notranslate"><span class="pre">SchemaProvider</span></code>, which can be registere [...]
+<p>In the following example, we’ll implement an in memory catalog, starting with the <code class="docutils literal notranslate"><span class="pre">SchemaProvider</span></code> trait as we need one to register with the <code class="docutils literal notranslate"><span class="pre">CatalogProvider</span></code>. Finally we will implement <code class="docutils literal notranslate"><span class="pre">CatalogProviderList</span></code> to register the <code class="docutils literal notranslate"><sp [...]
</section>
<section id="implementing-memoryschemaprovider">
<h2>Implementing <code class="docutils literal notranslate"><span class="pre">MemorySchemaProvider</span></code><a class="headerlink" href="#implementing-memoryschemaprovider" title="Link to this heading">¶</a></h2>
@@ -583,16 +583,16 @@
</div>
<p>Again, this is fairly straightforward, as there’s an underlying data structure to store the state, via key-value pairs.</p>
</section>
-<section id="implementing-memorycataloglist">
-<h2>Implementing <code class="docutils literal notranslate"><span class="pre">MemoryCatalogList</span></code><a class="headerlink" href="#implementing-memorycataloglist" title="Link to this heading">¶</a></h2>
-<div class="highlight-rust notranslate"><div class="highlight"><pre><span></span><span class="k">pub</span><span class="w"> </span><span class="k">struct</span> <span class="nc">MemoryCatalogList</span><span class="w"> </span><span class="p">{</span>
+<section id="implementing-memorycatalogproviderlist">
+<h2>Implementing <code class="docutils literal notranslate"><span class="pre">MemoryCatalogProviderList</span></code><a class="headerlink" href="#implementing-memorycatalogproviderlist" title="Link to this heading">¶</a></h2>
+<div class="highlight-rust notranslate"><div class="highlight"><pre><span></span><span class="k">pub</span><span class="w"> </span><span class="k">struct</span> <span class="nc">MemoryCatalogProviderList</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="sd">/// Collection of catalogs containing schemas and ultimately TableProviders</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="n">catalogs</span>: <span class="nc">DashMap</span><span class="o"><</span><span class="nb">String</span><span class="p">,</span><span class="w"> </span><span class="n">Arc</span><span class="o"><</span><span class="k">dyn</span><span class="w"> </span><span class="n">CatalogProvider</span><span class="o">>></span><span class="p">,</span>
<span class="p">}</span>
</pre></div>
</div>
-<p>With that the <code class="docutils literal notranslate"><span class="pre">CatalogList</span></code> trait can be implemented.</p>
-<div class="highlight-rust notranslate"><div class="highlight"><pre><span></span><span class="k">impl</span><span class="w"> </span><span class="n">CatalogList</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">MemoryCatalogList</span><span class="w"> </span><span class="p">{</span>
+<p>With that the <code class="docutils literal notranslate"><span class="pre">CatalogProviderList</span></code> trait can be implemented.</p>
+<div class="highlight-rust notranslate"><div class="highlight"><pre><span></span><span class="k">impl</span><span class="w"> </span><span class="n">CatalogProviderList</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">MemoryCatalogProviderList</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">fn</span> <span class="nf">as_any</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-> <span class="kp">&</span><span class="nc">dyn</span><span class="w"> </span><span class="n">Any</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="bp">self</span>
<span class="w"> </span><span class="p">}</span>
@@ -624,7 +624,7 @@
<li><p>Implement the <code class="docutils literal notranslate"><span class="pre">TableProvider</span></code> trait to create a table provider, or use an existing one.</p></li>
<li><p>Implement the <code class="docutils literal notranslate"><span class="pre">SchemaProvider</span></code> trait to create a schema provider, or use an existing one.</p></li>
<li><p>Implement the <code class="docutils literal notranslate"><span class="pre">CatalogProvider</span></code> trait to create a catalog provider, or use an existing one.</p></li>
-<li><p>Implement the <code class="docutils literal notranslate"><span class="pre">CatalogList</span></code> trait to create a CatalogList, or use an existing one.</p></li>
+<li><p>Implement the <code class="docutils literal notranslate"><span class="pre">CatalogProviderList</span></code> trait to create a CatalogProviderList, or use an existing one.</p></li>
</ol>
</section>
</section>
diff --git a/searchindex.js b/searchindex.js
index 32f362de44..ab3a23ee39 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["contributor-guide/architecture", "contributor-guide/communication", "contributor-guide/index", "contributor-guide/quarterly_roadmap", "contributor-guide/roadmap", "contributor-guide/specification/index", "contributor-guide/specification/invariants", "contributor-guide/specification/output-field-name-semantic", "index", "library-user-guide/adding-udfs", "library-user-guide/building-logical-plans", "library-user-guide/catalogs", "library-user-guide/custom-tab [...]
\ No newline at end of file
+Search.setIndex({"docnames": ["contributor-guide/architecture", "contributor-guide/communication", "contributor-guide/index", "contributor-guide/quarterly_roadmap", "contributor-guide/roadmap", "contributor-guide/specification/index", "contributor-guide/specification/invariants", "contributor-guide/specification/output-field-name-semantic", "index", "library-user-guide/adding-udfs", "library-user-guide/building-logical-plans", "library-user-guide/catalogs", "library-user-guide/custom-tab [...]
\ No newline at end of file