You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by ry...@apache.org on 2021/04/15 11:20:00 UTC
[iceberg] branch master updated: Simplify Docs release by adding a
version template (#2433)
This is an automated email from the ASF dual-hosted git repository.
rymurr pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iceberg.git
The following commit(s) were added to refs/heads/master by this push:
new 8b8f0e9 Simplify Docs release by adding a version template (#2433)
8b8f0e9 is described below
commit 8b8f0e94de69807f7521ed53d237cc205bb18b22
Author: Ryan Murray <ry...@gmail.com>
AuthorDate: Thu Apr 15 13:19:53 2021 +0200
Simplify Docs release by adding a version template (#2433)
* Simplify Docs release by adding a version template
This uses `mkdocs-markdownextradata-plugin` to template things like
versions. Hopefully this will simplify updating docs on releases
* add licence header
---
site/README.md | 5 +----
site/docs/aws.md | 8 ++++----
site/docs/getting-started.md | 8 ++++----
site/docs/nessie.md | 2 +-
site/docs/releases.md | 16 ++++++++--------
site/docs/spark-configuration.md | 2 +-
site/mkdocs.yml | 4 ++++
site/requirements.txt | 23 +++++++++++++++++++++++
8 files changed, 46 insertions(+), 22 deletions(-)
diff --git a/site/README.md b/site/README.md
index bab0411..4fc7634 100644
--- a/site/README.md
+++ b/site/README.md
@@ -30,10 +30,7 @@ This directory contains the source for the Iceberg site.
The site is built using mkdocs. To install mkdocs and the theme, run:
```
-pip install mkdocs
-pip install mkdocs-cinder
-pip install mkdocs-redirects
-pip install pymdown-extensions
+pip install -r requirements.txt
```
### Local Changes
diff --git a/site/docs/aws.md b/site/docs/aws.md
index e9ba3e5..a127437 100644
--- a/site/docs/aws.md
+++ b/site/docs/aws.md
@@ -44,7 +44,7 @@ For example, to use AWS features with Spark 3 and AWS clients version 2.15.40, y
```sh
# add Iceberg dependency
-ICEBERG_VERSION=0.11.0
+ICEBERG_VERSION={{ versions.iceberg }}
DEPENDENCIES="org.apache.iceberg:iceberg-spark3-runtime:$ICEBERG_VERSION"
# add AWS dependnecy
@@ -76,7 +76,7 @@ To use AWS module with Flink, you can download the necessary dependencies and sp
```sh
# download Iceberg dependency
-ICEBERG_VERSION=0.11.0
+ICEBERG_VERSION={{ versions.iceberg }}
MAVEN_URL=https://repo1.maven.org/maven2
ICEBERG_MAVEN_URL=$MAVEN_URL/org/apache/iceberg
wget $ICEBERG_MAVEN_URL/iceberg-flink-runtime/$ICEBERG_VERSION/iceberg-flink-runtime-$ICEBERG_VERSION.jar
@@ -322,7 +322,7 @@ The Glue, S3 and DynamoDB clients are then initialized with the assume-role cred
Here is an example to start Spark shell with this client factory:
```shell
-spark-sql --packages org.apache.iceberg:iceberg-spark3-runtime:0.11.0,software.amazon.awssdk:bundle:2.15.40 \
+spark-sql --packages org.apache.iceberg:iceberg-spark3-runtime:{{ versions.iceberg }},software.amazon.awssdk:bundle:2.15.40 \
--conf spark.sql.catalog.my_catalog=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.my_catalog.warehouse=s3://my-bucket/my/key/prefix \
--conf spark.sql.catalog.my_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog \
@@ -345,7 +345,7 @@ You can use a [bootstrap action](https://docs.aws.amazon.com/emr/latest/Manageme
#!/bin/bash
AWS_SDK_VERSION=2.15.40
-ICEBERG_VERSION=0.11.0
+ICEBERG_VERSION={{ versions.iceberg }}
MAVEN_URL=https://repo1.maven.org/maven2
ICEBERG_MAVEN_URL=$MAVEN_URL/org/apache/iceberg
AWS_MAVEN_URL=$MAVEN_URL/software/amazon/awssdk
diff --git a/site/docs/getting-started.md b/site/docs/getting-started.md
index 83cd95a..e496bb6 100644
--- a/site/docs/getting-started.md
+++ b/site/docs/getting-started.md
@@ -19,18 +19,18 @@
## Using Iceberg in Spark 3
-The latest version of Iceberg is [0.11.0](./releases.md).
+The latest version of Iceberg is [{{ versions.iceberg }}](./releases.md).
To use Iceberg in a Spark shell, use the `--packages` option:
```sh
-spark-shell --packages org.apache.iceberg:iceberg-spark3-runtime:0.11.0
+spark-shell --packages org.apache.iceberg:iceberg-spark3-runtime:{{ versions.iceberg }}
```
!!! Note
If you want to include Iceberg in your Spark installation, add the [`iceberg-spark3-runtime` Jar][spark-runtime-jar] to Spark's `jars` folder.
-[spark-runtime-jar]: https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark3-runtime/0.11.0/iceberg-spark3-runtime-0.11.0.jar
+[spark-runtime-jar]: https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark3-runtime/{{ versions.iceberg }}/iceberg-spark3-runtime-{{ versions.iceberg }}.jar
### Adding catalogs
@@ -39,7 +39,7 @@ Iceberg comes with [catalogs](./spark-configuration.md#catalogs) that enable SQL
This command creates a path-based catalog named `local` for tables under `$PWD/warehouse` and adds support for Iceberg tables to Spark's built-in catalog:
```sh
-spark-sql --packages org.apache.iceberg:iceberg-spark3-runtime:0.11.0 \
+spark-sql --packages org.apache.iceberg:iceberg-spark3-runtime:{{ versions.iceberg }}\
--conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \
--conf spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog \
--conf spark.sql.catalog.spark_catalog.type=hive \
diff --git a/site/docs/nessie.md b/site/docs/nessie.md
index 2e2fea4..c4bfb5b 100644
--- a/site/docs/nessie.md
+++ b/site/docs/nessie.md
@@ -31,7 +31,7 @@ See [Project Nessie](https://projectnessie.org) for more information on Nessie.
The `iceberg-nessie` module is bundled with Spark and Flink runtimes for all versions from `0.11.0`. To get started
with nessie and iceberg simply add the iceberg runtime to your process. Eg: `spark-sql --packages
-org.apache.iceberg:iceberg-spark3-runtiume:0.11.0`.
+org.apache.iceberg:iceberg-spark3-runtiume:{{ versions.iceberg }}`.
## Nessie Catalog
diff --git a/site/docs/releases.md b/site/docs/releases.md
index d52a6be..76420f1 100644
--- a/site/docs/releases.md
+++ b/site/docs/releases.md
@@ -17,13 +17,13 @@
## Downloads
-The latest version of Iceberg is [0.11.0](https://github.com/apache/iceberg/releases/tag/apache-iceberg-0.11.0).
+The latest version of Iceberg is [{{ versions.iceberg }}](https://github.com/apache/iceberg/releases/tag/apache-iceberg-{{ versions.iceberg }}).
-* [0.11.0 source tar.gz](https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-0.11.0/apache-iceberg-0.11.0.tar.gz) -- [signature](https://downloads.apache.org/iceberg/apache-iceberg-0.11.0/apache-iceberg-0.11.0.tar.gz.asc) -- [sha512](https://downloads.apache.org/iceberg/apache-iceberg-0.11.0/apache-iceberg-0.11.0.tar.gz.sha512)
-* [0.11.0 Spark 3.0 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark3-runtime/0.11.0/iceberg-spark3-runtime-0.11.0.jar)
-* [0.11.0 Spark 2.4 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime/0.11.0/iceberg-spark-runtime-0.11.0.jar)
-* [0.11.0 Flink runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime/0.11.0/iceberg-flink-runtime-0.11.0.jar)
-* [0.11.0 Hive runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-hive-runtime/0.11.0/iceberg-hive-runtime-0.11.0.jar)
+* [{{ versions.iceberg }} source tar.gz](https://www.apache.org/dyn/closer.cgi/iceberg/apache-iceberg-{{ versions.iceberg }}/apache-iceberg-{{ versions.iceberg }}.tar.gz) -- [signature](https://downloads.apache.org/iceberg/apache-iceberg-{{ versions.iceberg }}/apache-iceberg-{{ versions.iceberg }}.tar.gz.asc) -- [sha512](https://downloads.apache.org/iceberg/apache-iceberg-{{ versions.iceberg }}/apache-iceberg-{{ versions.iceberg }}.tar.gz.sha512)
+* [{{ versions.iceberg }} Spark 3.0 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark3-runtime/{{ versions.iceberg }}/iceberg-spark3-runtime-{{ versions.iceberg }}.jar)
+* [{{ versions.iceberg }} Spark 2.4 runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime/{{ versions.iceberg }}/iceberg-spark-runtime-{{ versions.iceberg }}.jar)
+* [{{ versions.iceberg }} Flink runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-flink-runtime/{{ versions.iceberg }}/iceberg-flink-runtime-{{ versions.iceberg }}.jar)
+* [{{ versions.iceberg }} Hive runtime Jar](https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-hive-runtime/{{ versions.iceberg }}/iceberg-hive-runtime-{{ versions.iceberg }}.jar)
To use Iceberg in Spark, download the runtime Jar and add it to the jars folder of your Spark install. Use iceberg-spark3-runtime for Spark 3, and iceberg-spark-runtime for Spark 2.4.
@@ -35,7 +35,7 @@ To add a dependency on Iceberg in Gradle, add the following to `build.gradle`:
```
dependencies {
- compile 'org.apache.iceberg:iceberg-core:0.11.0'
+ compile 'org.apache.iceberg:iceberg-core:{{ versions.iceberg }}'
}
```
@@ -51,7 +51,7 @@ To add a dependency on Iceberg in Maven, add the following to your `pom.xml`:
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-core</artifactId>
- <version>0.11.0</version>
+ <version>{{ versions.iceberg }}</version>
</dependency>
...
</dependencies>
diff --git a/site/docs/spark-configuration.md b/site/docs/spark-configuration.md
index 5c67a77..0ae27ca 100644
--- a/site/docs/spark-configuration.md
+++ b/site/docs/spark-configuration.md
@@ -106,7 +106,7 @@ spark.sql.catalog.custom_prod.my-additional-catalog-config = my-value
### Catalogs in Spark 2.4
-When using Iceberg 0.11.0, Spark 2.4 can load tables from multiple Iceberg catalogs or from table locations.
+When using Iceberg 0.11.0 and later, Spark 2.4 can load tables from multiple Iceberg catalogs or from table locations.
Catalogs in 2.4 are configured just like catalogs in 3.0, but only Iceberg catalogs are supported.
diff --git a/site/mkdocs.yml b/site/mkdocs.yml
index 573452f..73f9167 100644
--- a/site/mkdocs.yml
+++ b/site/mkdocs.yml
@@ -28,8 +28,12 @@ theme: cinder
copyright: "Copyright 2018-2021 <a href='https://www.apache.org/'>The Apache Software Foundation</a><br />Apache Iceberg, Iceberg, Apache, the Apache feather logo, and the Apache Iceberg project logo are either registered<br />trademarks or trademarks of The Apache Software Foundation in the United States and other countries."
extra_css:
- css/extra.css
+extra:
+ versions:
+ iceberg: 0.11.1
plugins:
- redirects
+ - markdownextradata
markdown_extensions:
- toc:
permalink: True
diff --git a/site/requirements.txt b/site/requirements.txt
new file mode 100644
index 0000000..989d7d8
--- /dev/null
+++ b/site/requirements.txt
@@ -0,0 +1,23 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+mkdocs-cinder
+mkdocs-redirects
+pymdown-extensions
+mkdocs-markdownextradata-plugin