You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by ru...@apache.org on 2021/10/07 14:19:24 UTC

[iceberg] branch master updated: Doc: Update Nessie integration doc about Nessie SQL Extensions (#3233)

This is an automated email from the ASF dual-hosted git repository.

russellspitzer pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iceberg.git


The following commit(s) were added to refs/heads/master by this push:
     new 7b2daa2  Doc: Update Nessie integration doc about Nessie SQL Extensions (#3233)
7b2daa2 is described below

commit 7b2daa297de9658621eee17d955aa85d74d20379
Author: Ajantha Bhat <aj...@gmail.com>
AuthorDate: Thu Oct 7 19:49:11 2021 +0530

    Doc: Update Nessie integration doc about Nessie SQL Extensions (#3233)
---
 site/docs/nessie.md | 14 +++++++++++++-
 site/mkdocs.yml     |  1 +
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/site/docs/nessie.md b/site/docs/nessie.md
index 9050fea..3dbbd5b 100644
--- a/site/docs/nessie.md
+++ b/site/docs/nessie.md
@@ -33,6 +33,18 @@ The `iceberg-nessie` module is bundled with Spark and Flink runtimes for all ver
 with Nessie and Iceberg simply add the Iceberg runtime to your process. Eg: `spark-sql --packages
 org.apache.iceberg:iceberg-spark3-runtiume:{{ versions.iceberg }}`. 
 
+## Spark SQL Extensions
+
+From spark, Nessie SQL extensions can be used to manage the Nessie repo as shown below. 
+
+```
+bin/spark-sql 
+  --packages "org.apache.iceberg:iceberg-spark3-runtime:{{ versions.iceberg }},org.projectnessie:nessie-spark-extensions:{{ versions.nessie }}"
+  --conf spark.sql.extensions="org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions"
+  --conf <other settings>
+```
+Please refer [Nessie SQL extension document](https://projectnessie.org/tools/sql/) to learn more about it.
+
 ## Nessie Catalog
 
 One major feature introduced in release `0.11.0` is the ability to easily interact with a [Custom
@@ -63,6 +75,7 @@ conf.set("spark.sql.catalog.nessie.uri", "http://localhost:19120/api/v1")
 conf.set("spark.sql.catalog.nessie.ref", "main")
 conf.set("spark.sql.catalog.nessie.catalog-impl", "org.apache.iceberg.nessie.NessieCatalog")
 conf.set("spark.sql.catalog.nessie", "org.apache.iceberg.spark.SparkCatalog")
+conf.set("spark.sql.extensions", "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions")
 ```
 This is how it looks in Flink via the Python API (additional details can be found [here](flink.md)):
 ```python
@@ -140,5 +153,4 @@ for different examples of Nessie and Iceberg in action together.
 
 ## Future Improvements
 
-* Nessie SQL extensions to manage the Nessie repo from Spark SQL
 * Iceberg multi-table transactions. Changes to multiple Iceberg tables in the same transaction, isolation levels etc
diff --git a/site/mkdocs.yml b/site/mkdocs.yml
index 3b428be..c8b3e6b 100644
--- a/site/mkdocs.yml
+++ b/site/mkdocs.yml
@@ -32,6 +32,7 @@ extra_css:
 extra:
   versions:
     iceberg: 0.12.0
+    nessie: 0.9.2
 plugins:
   - redirects:
       redirect_maps: