You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by ru...@apache.org on 2021/10/07 14:19:24 UTC
[iceberg] branch master updated: Doc: Update Nessie integration doc
about Nessie SQL Extensions (#3233)
This is an automated email from the ASF dual-hosted git repository.
russellspitzer pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iceberg.git
The following commit(s) were added to refs/heads/master by this push:
new 7b2daa2 Doc: Update Nessie integration doc about Nessie SQL Extensions (#3233)
7b2daa2 is described below
commit 7b2daa297de9658621eee17d955aa85d74d20379
Author: Ajantha Bhat <aj...@gmail.com>
AuthorDate: Thu Oct 7 19:49:11 2021 +0530
Doc: Update Nessie integration doc about Nessie SQL Extensions (#3233)
---
site/docs/nessie.md | 14 +++++++++++++-
site/mkdocs.yml | 1 +
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/site/docs/nessie.md b/site/docs/nessie.md
index 9050fea..3dbbd5b 100644
--- a/site/docs/nessie.md
+++ b/site/docs/nessie.md
@@ -33,6 +33,18 @@ The `iceberg-nessie` module is bundled with Spark and Flink runtimes for all ver
with Nessie and Iceberg simply add the Iceberg runtime to your process. Eg: `spark-sql --packages
org.apache.iceberg:iceberg-spark3-runtiume:{{ versions.iceberg }}`.
+## Spark SQL Extensions
+
+From spark, Nessie SQL extensions can be used to manage the Nessie repo as shown below.
+
+```
+bin/spark-sql
+ --packages "org.apache.iceberg:iceberg-spark3-runtime:{{ versions.iceberg }},org.projectnessie:nessie-spark-extensions:{{ versions.nessie }}"
+ --conf spark.sql.extensions="org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions"
+ --conf <other settings>
+```
+Please refer [Nessie SQL extension document](https://projectnessie.org/tools/sql/) to learn more about it.
+
## Nessie Catalog
One major feature introduced in release `0.11.0` is the ability to easily interact with a [Custom
@@ -63,6 +75,7 @@ conf.set("spark.sql.catalog.nessie.uri", "http://localhost:19120/api/v1")
conf.set("spark.sql.catalog.nessie.ref", "main")
conf.set("spark.sql.catalog.nessie.catalog-impl", "org.apache.iceberg.nessie.NessieCatalog")
conf.set("spark.sql.catalog.nessie", "org.apache.iceberg.spark.SparkCatalog")
+conf.set("spark.sql.extensions", "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions")
```
This is how it looks in Flink via the Python API (additional details can be found [here](flink.md)):
```python
@@ -140,5 +153,4 @@ for different examples of Nessie and Iceberg in action together.
## Future Improvements
-* Nessie SQL extensions to manage the Nessie repo from Spark SQL
* Iceberg multi-table transactions. Changes to multiple Iceberg tables in the same transaction, isolation levels etc
diff --git a/site/mkdocs.yml b/site/mkdocs.yml
index 3b428be..c8b3e6b 100644
--- a/site/mkdocs.yml
+++ b/site/mkdocs.yml
@@ -32,6 +32,7 @@ extra_css:
extra:
versions:
iceberg: 0.12.0
+ nessie: 0.9.2
plugins:
- redirects:
redirect_maps: