[GitHub] [hudi] alexeykudinkin commented on a change in pull request #5013: [HUDI-3593] Restore TypedProperties and flush checksum in table config

[GitBox <gi...@apache.org>]

alexeykudinkin commented on a change in pull request #5013:
URL: https://github.com/apache/hudi/pull/5013#discussion_r823993627



##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -318,11 +337,13 @@ private static void modify(FileSystem fs, Path metadataFolder, Properties modify
         modifyFn.accept(props, modifyProps);
         if (props.containsKey(TABLE_CHECKSUM.key()) && validateChecksum(props)) {

Review comment:
       Let's extract this whole conditional as a `persist` method (for ex), since it's pretty much replicated in 3 places 

##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -221,9 +220,12 @@ public HoodieTableConfig(FileSystem fs, String metaPath, String payloadClassName
         // FIXME(vc): wonder if this can be removed. Need to look into history.
         try (FSDataOutputStream outputStream = fs.create(propertyPath)) {
           if (!isValidChecksum()) {
-            setValue(TABLE_CHECKSUM, String.valueOf(generateChecksum(props)));
+            Properties propsWithChecksum = getOrderedPropertiesWithTableChecksum(props);
+            propsWithChecksum.store(outputStream, "Properties saved on " + new Date(System.currentTimeMillis()));

Review comment:
       Let's use `Instant.now` instead (Date is a legacy API)

##########
File path: hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
##########
@@ -233,6 +235,23 @@ public HoodieTableConfig(FileSystem fs, String metaPath, String payloadClassName
         "hoodie.properties file seems invalid. Please check for left over `.updated` files if any, manually copy it to hoodie.properties and retry");
   }
 
+  private static Properties getOrderedPropertiesWithTableChecksum(Properties props) {
+    LinkedHashMap<String, String> propsMap = getOrderedPropertiesMap(props);
+    propsMap.put(TABLE_CHECKSUM.key(), String.valueOf(generateChecksum(props)));
+    Properties orderedProps = new Properties();

Review comment:
       This will destroy the ordering, right? You need to control the iteration during `store`, you to create class that is similar to how this was handled in `TypedProperties`
   
   ```
   class OrderedProperties extends Properties {
     LinkedHashSet hashSet;
     
     Enumeration<> keys() { hashSet.keys() }
   }
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org