You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/02/09 17:49:11 UTC
[GitHub] [iceberg] SinghAsDev commented on a change in pull request #4011: Allow table defaults to be configured and/ or enforced at catalog level using catalog properties.
SinghAsDev commented on a change in pull request #4011:
URL: https://github.com/apache/iceberg/pull/4011#discussion_r802935212
##########
File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java
##########
@@ -238,4 +249,63 @@ protected static String fullTableName(String catalogName, TableIdentifier identi
return sb.toString();
}
+
+ @Override
+ public Table createTable(
+ TableIdentifier identifier,
+ Schema schema,
+ PartitionSpec spec,
+ String location,
+ Map<String, String> properties) {
+ return buildTable(identifier, schema)
+ .withPartitionSpec(spec)
+ .withLocation(location)
+ .withProperties(updateTableProperties(properties))
+ .create();
+ }
+
+ /**
+ * Get default table properties set at Catalog level through catalog properties.
+ *
+ * @return default table properties specified in catalog properties
+ */
+ private Map<String, String> tableDefaultProperties() {
+ Map<String, String> props = catalogProps == null ? Collections.emptyMap() : catalogProps;
+
+ return props.entrySet().stream()
+ .filter(e -> e.getKey().toLowerCase(Locale.ROOT).startsWith(CatalogProperties.TABLE_DEFAULT_PREFIX))
+ .collect(Collectors.toMap(e -> e.getKey().replace(CatalogProperties.TABLE_DEFAULT_PREFIX, ""),
+ Map.Entry::getValue));
+ }
+
+ /**
+ * Get table properties that are enforced at Catalog level through catalog properties.
+ *
+ * @return default table properties enforced through catalog properties
+ */
+ private Map<String, String> tableOverrideProperties() {
+ Map<String, String> props = catalogProps == null ? Collections.emptyMap() : catalogProps;
+
+ return props.entrySet().stream()
+ .filter(e -> e.getKey().toLowerCase(Locale.ROOT).startsWith(CatalogProperties.TABLE_OVERRIDE_PREFIX))
+ .collect(Collectors.toMap(e -> e.getKey().replace(CatalogProperties.TABLE_OVERRIDE_PREFIX, ""),
+ Map.Entry::getValue));
+ }
+
+ /**
+ * Return updated table properties with table properties defaults and enforcements set at Catalog level through
+ * catalog properties.
+ *
+ * @return updated table properties with defaults and enforcements set at Catalog level
+ */
+ private Map<String, String> updateTableProperties(Map<String, String> tableProperties) {
Review comment:
Hey @kbendick , thanks, I was having trouble coming up with a good name 😀. maybe `updateWithCatalogDerivedTableProps`? `combine` makes me think we are only merging (and not overriding), but we are.
##########
File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java
##########
@@ -238,4 +249,63 @@ protected static String fullTableName(String catalogName, TableIdentifier identi
return sb.toString();
}
+
+ @Override
+ public Table createTable(
+ TableIdentifier identifier,
+ Schema schema,
+ PartitionSpec spec,
+ String location,
+ Map<String, String> properties) {
+ return buildTable(identifier, schema)
+ .withPartitionSpec(spec)
+ .withLocation(location)
+ .withProperties(updateTableProperties(properties))
+ .create();
+ }
+
+ /**
+ * Get default table properties set at Catalog level through catalog properties.
+ *
+ * @return default table properties specified in catalog properties
+ */
+ private Map<String, String> tableDefaultProperties() {
+ Map<String, String> props = catalogProps == null ? Collections.emptyMap() : catalogProps;
+
+ return props.entrySet().stream()
+ .filter(e -> e.getKey().toLowerCase(Locale.ROOT).startsWith(CatalogProperties.TABLE_DEFAULT_PREFIX))
+ .collect(Collectors.toMap(e -> e.getKey().replace(CatalogProperties.TABLE_DEFAULT_PREFIX, ""),
+ Map.Entry::getValue));
+ }
+
+ /**
+ * Get table properties that are enforced at Catalog level through catalog properties.
+ *
+ * @return default table properties enforced through catalog properties
+ */
+ private Map<String, String> tableOverrideProperties() {
+ Map<String, String> props = catalogProps == null ? Collections.emptyMap() : catalogProps;
+
+ return props.entrySet().stream()
+ .filter(e -> e.getKey().toLowerCase(Locale.ROOT).startsWith(CatalogProperties.TABLE_OVERRIDE_PREFIX))
+ .collect(Collectors.toMap(e -> e.getKey().replace(CatalogProperties.TABLE_OVERRIDE_PREFIX, ""),
+ Map.Entry::getValue));
+ }
+
+ /**
+ * Return updated table properties with table properties defaults and enforcements set at Catalog level through
+ * catalog properties.
+ *
+ * @return updated table properties with defaults and enforcements set at Catalog level
+ */
+ private Map<String, String> updateTableProperties(Map<String, String> tableProperties) {
Review comment:
Thanks, this makes sense.
##########
File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java
##########
@@ -238,4 +249,63 @@ protected static String fullTableName(String catalogName, TableIdentifier identi
return sb.toString();
}
+
+ @Override
+ public Table createTable(
Review comment:
You are right, this is not needed. Removing it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org