You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by bl...@apache.org on 2021/06/15 17:20:22 UTC
[iceberg] branch master updated: Docs: Add cache-enabled to catalog
property list (#2648)
This is an automated email from the ASF dual-hosted git repository.
blue pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iceberg.git
The following commit(s) were added to refs/heads/master by this push:
new a6b0f42 Docs: Add cache-enabled to catalog property list (#2648)
a6b0f42 is described below
commit a6b0f4240c7d7d94fa5f6d0cbf88332610b10d52
Author: jshmchenxi <ch...@qiyi.com>
AuthorDate: Wed Jun 16 01:20:08 2021 +0800
Docs: Add cache-enabled to catalog property list (#2648)
---
flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java | 1 +
site/docs/flink.md | 1 +
site/docs/spark-configuration.md | 1 +
spark3/src/main/java/org/apache/iceberg/spark/SparkCatalog.java | 1 +
4 files changed, 4 insertions(+)
diff --git a/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java b/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java
index 406c979..e95e258 100644
--- a/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java
+++ b/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java
@@ -51,6 +51,7 @@ import org.apache.iceberg.relocated.com.google.common.collect.Maps;
* <li><code>warehouse</code> - the warehouse path (Hadoop catalog only)</li>
* <li><code>default-database</code> - a database name to use as the default</li>
* <li><code>base-namespace</code> - a base namespace as the prefix for all databases (Hadoop catalog only)</li>
+ * <li><code>cache-enabled</code> - whether to enable catalog cache</li>
* </ul>
* <p>
* To use a custom catalog that is not a Hive or Hadoop catalog, extend this class and override
diff --git a/site/docs/flink.md b/site/docs/flink.md
index 8836154..34ee502 100644
--- a/site/docs/flink.md
+++ b/site/docs/flink.md
@@ -196,6 +196,7 @@ CREATE CATALOG hive_catalog WITH (
* `property-version`: Version number to describe the property version. This property can be used for backwards compatibility in case the property format changes. The current property version is `1`. (Optional)
* `warehouse`: The Hive warehouse location, users should specify this path if neither set the `hive-conf-dir` to specify a location containing a `hive-site.xml` configuration file nor add a correct `hive-site.xml` to classpath.
* `hive-conf-dir`: Path to a directory containing a `hive-site.xml` configuration file which will be used to provide custom Hive configuration values. The value of `hive.metastore.warehouse.dir` from `<hive-conf-dir>/hive-site.xml` (or hive configure file from classpath) will be overwrote with the `warehouse` value if setting both `hive-conf-dir` and `warehouse` when creating iceberg catalog.
+* `cache-enabled`: Whether to enable catalog cache, default value is `true`
### Hadoop catalog
diff --git a/site/docs/spark-configuration.md b/site/docs/spark-configuration.md
index 0ae27ca..dd633f1 100644
--- a/site/docs/spark-configuration.md
+++ b/site/docs/spark-configuration.md
@@ -59,6 +59,7 @@ Both catalogs are configured using properties nested under the catalog name. Com
| spark.sql.catalog._catalog-name_.default-namespace | default | The default current namespace for the catalog |
| spark.sql.catalog._catalog-name_.uri | thrift://host:port | Metastore connect URI; default from `hive-site.xml` |
| spark.sql.catalog._catalog-name_.warehouse | hdfs://nn:8020/warehouse/path | Base path for the warehouse directory |
+| spark.sql.catalog._catalog-name_.cache-enabled | `true` or `false` | Whether to enable catalog cache, default value is `true` |
Additional properties can be found in common [catalog configuration](./configuration.md#catalog-properties).
diff --git a/spark3/src/main/java/org/apache/iceberg/spark/SparkCatalog.java b/spark3/src/main/java/org/apache/iceberg/spark/SparkCatalog.java
index ac7e6e9..e8c7db7 100644
--- a/spark3/src/main/java/org/apache/iceberg/spark/SparkCatalog.java
+++ b/spark3/src/main/java/org/apache/iceberg/spark/SparkCatalog.java
@@ -72,6 +72,7 @@ import org.apache.spark.sql.util.CaseInsensitiveStringMap;
* <li><code>uri</code> - the Hive Metastore URI (Hive catalog only)</li>
* <li><code>warehouse</code> - the warehouse path (Hadoop catalog only)</li>
* <li><code>default-namespace</code> - a namespace to use as the default</li>
+ * <li><code>cache-enabled</code> - whether to enable catalog cache</li>
* </ul>
* <p>
* To use a custom catalog that is not a Hive or Hadoop catalog, extend this class and override