You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by op...@apache.org on 2021/08/20 12:27:24 UTC

[iceberg] branch master updated: Doc: Improve the flink shell docs (#2920)

This is an automated email from the ASF dual-hosted git repository.

openinx pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iceberg.git


The following commit(s) were added to refs/heads/master by this push:
     new eaa2abf  Doc:  Improve the flink shell docs (#2920)
eaa2abf is described below

commit eaa2abf9a339d3e1997cdd9c5a75899f844c61ba
Author: mikewu <xi...@gmail.com>
AuthorDate: Fri Aug 20 20:27:12 2021 +0800

    Doc:  Improve the flink shell docs (#2920)
---
 site/docs/flink.md | 28 ++++++++++++++++++++++------
 1 file changed, 22 insertions(+), 6 deletions(-)

diff --git a/site/docs/flink.md b/site/docs/flink.md
index 673057d..da48161 100644
--- a/site/docs/flink.md
+++ b/site/docs/flink.md
@@ -44,8 +44,11 @@ To create iceberg table in flink, we recommend to use [Flink SQL Client](https:/
 Step.1 Downloading the flink 1.11.x binary package from the apache flink [download page](https://flink.apache.org/downloads.html). We now use scala 2.12 to archive the apache iceberg-flink-runtime jar, so it's recommended to use flink 1.11 bundled with scala 2.12.
 
 ```bash
-wget https://downloads.apache.org/flink/flink-1.11.1/flink-1.11.1-bin-scala_2.12.tgz
-tar xzvf flink-1.11.1-bin-scala_2.12.tgz
+FLINK_VERSION=1.11.1
+SCALA_VERSION=2.12
+APACHE_FLINK_URL=archive.apache.org/dist/flink/
+wget ${APACHE_FLINK_URL}/flink-${FLINK_VERSION}/flink-${FLINK_VERSION}-bin-scala_${SCALA_VERSION}.tgz
+tar xzvf flink-${FLINK_VERSION}-bin-scala_${SCALA_VERSION}.tgz
 ```
 
 Step.2 Start a standalone flink cluster within hadoop environment.
@@ -78,12 +81,25 @@ as the following:
 # HADOOP_HOME is your hadoop root directory after unpack the binary package.
 export HADOOP_CLASSPATH=`$HADOOP_HOME/bin/hadoop classpath`
 
-# wget the flink-sql-connector-hive-2.3.6_2.11-1.11.0.jar from the above bundled jar URL firstly.
+# download Iceberg dependency
+ICEBERG_VERSION=0.11.1
+MAVEN_URL=https://repo1.maven.org/maven2
+ICEBERG_MAVEN_URL=${MAVEN_URL}/org/apache/iceberg
+ICEBERG_PACKAGE=iceberg-flink-runtime
+wget ${ICEBERG_MAVEN_URL}/${ICEBERG_PACKAGE}/${ICEBERG_VERSION}/${ICEBERG_PACKAGE}-${ICEBERG_VERSION}.jar
+
+# download the flink-sql-connector-hive-${HIVE_VERSION}_${SCALA_VERSION}-${FLINK_VERSION}.jar
+HIVE_VERSION=2.3.6
+SCALA_VERSION=2.11
+FLINK_VERSION=1.11.0
+FLINK_CONNECTOR_URL=${MAVEN_URL}/org/apache/flink
+FLINK_CONNECTOR_PACKAGE=flink-sql-connector-hive
+wget ${FLINK_CONNECTOR_URL}/${FLINK_CONNECTOR_PACKAGE}-${HIVE_VERSION}_${SCALA_VERSION}/${FLINK_VERSION}/${FLINK_CONNECTOR_PACKAGE}-${HIVE_VERSION}_${SCALA_VERSION}-${FLINK_VERSION}.jar
 
 # open the SQL client.
-./bin/sql-client.sh embedded \
-    -j <flink-runtime-directory>/iceberg-flink-runtime-xxx.jar \
-    -j <hive-bundlded-jar-directory>/flink-sql-connector-hive-2.3.6_2.11-1.11.0.jar \
+/path/to/bin/sql-client.sh embedded \
+    -j ${ICEBERG_PACKAGE}-${ICEBERG_VERSION}.jar \
+    -j ${FLINK_CONNECTOR_PACKAGE}-${HIVE_VERSION}_${SCALA_VERSION}-${FLINK_VERSION}.jar \
     shell
 ```
 ## Preparation when using Flink's Python API