[GitHub] [iceberg] singhpk234 commented on a change in pull request #4011: Allow table defaults to be configured and/ or enforced at catalog level using catalog properties.

[GitBox <gi...@apache.org>]

singhpk234 commented on a change in pull request #4011:
URL: https://github.com/apache/iceberg/pull/4011#discussion_r807620069



##########
File path: core/src/main/java/org/apache/iceberg/util/PropertyUtil.java
##########
@@ -70,4 +72,16 @@ public static String propertyAsString(Map<String, String> properties,
     }
     return defaultValue;
   }
+
+  public static Map<String, String> propertiesWithPrefix(Map<String, String> properties,
+                                                         String prefix) {

Review comment:
       [question] since this is a public should we check `prefix != null` as replace requires the target CharSequence to be NonNull ? 

##########
File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java
##########
@@ -214,6 +224,32 @@ private Transaction newReplaceTableTransaction(boolean orCreate) {
         return Transactions.replaceTableTransaction(identifier.toString(), ops, metadata);
       }
     }
+
+    /**
+     * Get default table properties set at Catalog level through catalog properties.
+     *
+     * @return default table properties specified in catalog properties
+     */
+    private Map<String, String> tableDefaultProperties() {
+      if (catalogProps == null || catalogProps.isEmpty()) {
+        return Collections.emptyMap();
+      }
+
+      return PropertyUtil.propertiesWithPrefix(catalogProps, CatalogProperties.TABLE_DEFAULT_PREFIX);
+    }
+
+    /**
+     * Get table properties that are enforced at Catalog level through catalog properties.
+     *
+     * @return default table properties enforced through catalog properties
+     */
+    private Map<String, String> tableOverrideProperties() {
+      if (catalogProps == null || catalogProps.isEmpty()) {
+        return Collections.emptyMap();
+      }
+
+      return PropertyUtil.propertiesWithPrefix(catalogProps, CatalogProperties.TABLE_OVERRIDE_PREFIX);
+    }

Review comment:
       [minor] how about making a single function and taking Prefix as argument, as the code skeleton is almost same
   
   some thing like : 
   ```
   private Map<String, String> extendedTableProperties(String prefix) {
        if (catalogProps == null || catalogProps.isEmpty()) {
          return Collections.emptyMap();
        }
   
       return PropertyUtil.propertiesWithPrefix(catalogProps, prefix);
   }
   ```

##########
File path: core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java
##########
@@ -98,6 +98,8 @@ public HadoopCatalog() {
 
   @Override
   public void initialize(String name, Map<String, String> properties) {
+    super.initialize(name, properties);

Review comment:
       [question] Any reason we didn't do it for glue catalog

##########
File path: core/src/test/java/org/apache/iceberg/hadoop/TestHadoopCatalog.java
##########
@@ -547,4 +548,60 @@ private static void addVersionsToTable(Table table) {
     table.newAppend().appendFile(dataFile1).commit();
     table.newAppend().appendFile(dataFile2).commit();
   }
+

Review comment:
       how about adding a test case when override / default are on the same key 
   same key is also passed with property




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org