You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by bl...@apache.org on 2020/11/20 01:11:37 UTC
[iceberg] branch asf-site updated: Deployed 953a7fdd7 with MkDocs
version: 1.0.4
This is an automated email from the ASF dual-hosted git repository.
blue pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/iceberg.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 84ee420 Deployed 953a7fdd7 with MkDocs version: 1.0.4
84ee420 is described below
commit 84ee4201fbb19926ecfdddf3073e4ba377ba4363
Author: Ryan Blue <bl...@apache.org>
AuthorDate: Thu Nov 19 17:11:24 2020 -0800
Deployed 953a7fdd7 with MkDocs version: 1.0.4
---
hive/index.html | 29 +++++++++++++++++++++++++++++
index.html | 2 +-
sitemap.xml | 50 +++++++++++++++++++++++++-------------------------
sitemap.xml.gz | Bin 227 -> 226 bytes
4 files changed, 55 insertions(+), 26 deletions(-)
diff --git a/hive/index.html b/hive/index.html
index 8c3515f..26e7c85 100644
--- a/hive/index.html
+++ b/hive/index.html
@@ -404,6 +404,35 @@ LOCATION 'hdfs://some_bucket/some_path/table_a';
<pre><code class="sql">SELECT * from table_a;
</code></pre>
+<h4 id="using-hive-catalog">Using Hive Catalog<a class="headerlink" href="#using-hive-catalog" title="Permanent link">¶</a></h4>
+<p>Iceberg tables created using <code>HiveCatalog</code> are automatically registered with Hive.</p>
+<h5 id="create-an-iceberg-table_1">Create an Iceberg table<a class="headerlink" href="#create-an-iceberg-table_1" title="Permanent link">¶</a></h5>
+<p>The first step is to create an Iceberg table using the Spark/Java/Python API and <code>HiveCatalog</code>. For the purposes of this documentation we will assume that the table is called <code>table_b</code> and that the table location is <code>s3://some_path/table_b</code>. In order for Iceberg to correctly set up the Hive table for querying some configuration values need to be set, the two options for this are described below - you can use either or the other depending on your use case.</p>
+<h5 id="hive-configuration">Hive Configuration<a class="headerlink" href="#hive-configuration" title="Permanent link">¶</a></h5>
+<p>The value <code>iceberg.engine.hive.enabled</code> needs to be set to <code>true</code> and added to the Hive configuration file on the classpath of the application creating the table. This can be done by modifying the relevant <code>hive-site.xml</code>. Alternatively this can done programatically like so:</p>
+<pre><code class="java">Configuration hadoopConfiguration = spark.sparkContext().hadoopConfiguration();
+hadoopConfiguration.set(ConfigProperties.ENGINE_HIVE_ENABLED, "true"); //iceberg.engine.hive.enabled=true
+HiveCatalog catalog = new HiveCatalog(hadoopConfiguration);
+...
+catalog.createTable(tableId, schema, spec);
+</code></pre>
+
+<h5 id="table-property-configuration">Table Property Configuration<a class="headerlink" href="#table-property-configuration" title="Permanent link">¶</a></h5>
+<p>The property <code>engine.hive.enabled</code> needs to be set to <code>true</code> and added to the table properties when creating the Iceberg table. This can be done like so:</p>
+<pre><code class="java"> Map<String, String> tableProperties = new HashMap<String, String>();
+ tableProperties.put(TableProperties.ENGINE_HIVE_ENABLED, "true"); //engine.hive.enabled=true
+ catalog.createTable(tableId, schema, spec, tableProperties);
+</code></pre>
+
+<h4 id="query-the-iceberg-table-via-hive_1">Query the Iceberg table via Hive<a class="headerlink" href="#query-the-iceberg-table-via-hive_1" title="Permanent link">¶</a></h4>
+<p>In order to query a Hive table created by either of the HiveCatalog methods described above you need to first set a Hive configuration value like so:</p>
+<pre><code class="sql">SET iceberg.mr.catalog=hive;
+</code></pre>
+
+<p>You should now be able to issue Hive SQL <code>SELECT</code> queries using the above table and see the results returned from the underlying Iceberg table. Both the Map Reduce and Tez query execution engines are supported.</p>
+<pre><code class="sql">SELECT * from table_b;
+</code></pre>
+
<h3 id="features">Features<a class="headerlink" href="#features" title="Permanent link">¶</a></h3>
<h4 id="predicate-pushdown">Predicate pushdown<a class="headerlink" href="#predicate-pushdown" title="Permanent link">¶</a></h4>
<p>Pushdown of the Hive SQL <code>WHERE</code> clause has been implemented so that these filters are used at the Iceberg TableScan level as well as by the Parquet and ORC Readers.</p></div>
diff --git a/index.html b/index.html
index b684abb..001646a 100644
--- a/index.html
+++ b/index.html
@@ -472,5 +472,5 @@
<!--
MkDocs version : 1.0.4
-Build Date UTC : 2020-11-16 23:54:03
+Build Date UTC : 2020-11-20 01:11:24
-->
diff --git a/sitemap.xml b/sitemap.xml
index 11bbb5b..bcc4634 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -2,77 +2,77 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
@@ -87,12 +87,12 @@
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
@@ -102,42 +102,42 @@
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
<loc>None</loc>
- <lastmod>2020-11-16</lastmod>
+ <lastmod>2020-11-19</lastmod>
<changefreq>daily</changefreq>
</url>
<url>
diff --git a/sitemap.xml.gz b/sitemap.xml.gz
index 74a44e0..6ed28ca 100644
Binary files a/sitemap.xml.gz and b/sitemap.xml.gz differ