[GitHub] [hudi] yihua commented on a change in pull request #4975: [HUDI-3404] Automatically adjust write configs based on metadata table and write concurrency mode

[GitBox <gi...@apache.org>]

yihua commented on a change in pull request #4975:
URL: https://github.com/apache/hudi/pull/4975#discussion_r828548624



##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
##########
@@ -2375,19 +2375,49 @@ protected void setDefaults() {
           HoodieLayoutConfig.newBuilder().fromProperties(writeConfig.getProps()).build());
       writeConfig.setDefaultValue(TIMELINE_LAYOUT_VERSION_NUM, String.valueOf(TimelineLayoutVersion.CURR_VERSION));
 
-      // Async table services can update the metadata table and a lock provider is
-      // needed to guard against any concurrent table write operations. If user has
-      // not configured any lock provider, let's use the InProcess lock provider.
+      autoAdjustConfigsForConcurrencyMode();
+    }
+
+    private void autoAdjustConfigsForConcurrencyMode() {
+      boolean isMetadataTableEnabled = writeConfig.getBoolean(HoodieMetadataConfig.ENABLE);
       final TypedProperties writeConfigProperties = writeConfig.getProps();
       final boolean isLockProviderPropertySet = writeConfigProperties.containsKey(HoodieLockConfig.LOCK_PROVIDER_CLASS_NAME)
           || writeConfigProperties.containsKey(HoodieLockConfig.LOCK_PROVIDER_CLASS_PROP);
+      
       if (!isLockConfigSet) {
         HoodieLockConfig.Builder lockConfigBuilder = HoodieLockConfig.newBuilder().fromProperties(writeConfig.getProps());
-        if (!isLockProviderPropertySet && writeConfig.areAnyTableServicesAsync()) {
-          lockConfigBuilder.withLockProvider(InProcessLockProvider.class);
-        }
         writeConfig.setDefault(lockConfigBuilder.build());
       }
+
+      if (isMetadataTableEnabled) {
+        // When metadata table is enabled, optimistic concurrency control must be used for
+        // single writer with async table services.
+        // Async table services can update the metadata table and a lock provider is
+        // needed to guard against any concurrent table write operations. If user has
+        // not configured any lock provider, let's use the InProcess lock provider.
+        boolean areTableServicesEnabled = writeConfig.areTableServicesEnabled();
+        boolean areAsyncTableServicesEnabled = writeConfig.areAnyTableServicesAsync();
+
+        if (!isLockProviderPropertySet && areTableServicesEnabled && areAsyncTableServicesEnabled) {
+          // This is targeted at Single writer with async table services

Review comment:
       We need it.  `areTableServicesEnabled` and `areAsyncTableServicesEnabled` are completely independent based on the configs.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org