[GitHub] [hudi] nsivabalan commented on a diff in pull request #8771: [HUDI-6245] Automatically downgrade table version of metadata table

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on code in PR #8771:
URL: https://github.com/apache/hudi/pull/8771#discussion_r1199675883


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -252,8 +253,9 @@ private void enablePartition(final MetadataPartitionType partitionType, final Ho
    * @param writeConfig                {@code HoodieWriteConfig} of the main dataset writer
    * @param failedWritesCleaningPolicy Cleaning policy on failed writes
    */
-  private HoodieWriteConfig createMetadataWriteConfig(
+  public static HoodieWriteConfig createMetadataWriteConfig(

Review Comment:
   can we move static methods to HoodieTableMetadataUtil ? 



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -105,6 +108,25 @@ public boolean needsUpgradeOrDowngrade(HoodieTableVersion toVersion) {
    * @param instantTime current instant time that should not be touched.
    */
   public void run(HoodieTableVersion toVersion, String instantTime) {
+    // Change metadata table version automatically
+    if (toVersion.versionCode() >= HoodieTableVersion.FOUR.versionCode()) {
+      String metadataTablePath = HoodieTableMetadata.getMetadataTableBasePath(
+          metaClient.getBasePathV2().toString());
+      try {
+        if (metaClient.getFs().exists(new Path(metadataTablePath))) {
+          HoodieTableMetaClient mdtMetaClient = HoodieTableMetaClient.builder()
+              .setConf(metaClient.getHadoopConf()).setBasePath(metadataTablePath).build();
+          HoodieWriteConfig mdtWriteConfig = HoodieBackedTableMetadataWriter.createMetadataWriteConfig(
+              config, HoodieFailedWritesCleaningPolicy.LAZY);

Review Comment:
   So, this takes care of both upgrade and downgrade right. i.e. upgrade will upgrade both data table and mdt table and downgrade will downgrade both ?



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/UpgradeDowngrade.java:
##########
@@ -105,6 +108,25 @@ public boolean needsUpgradeOrDowngrade(HoodieTableVersion toVersion) {
    * @param instantTime current instant time that should not be touched.
    */
   public void run(HoodieTableVersion toVersion, String instantTime) {
+    // Change metadata table version automatically
+    if (toVersion.versionCode() >= HoodieTableVersion.FOUR.versionCode()) {
+      String metadataTablePath = HoodieTableMetadata.getMetadataTableBasePath(
+          metaClient.getBasePathV2().toString());
+      try {
+        if (metaClient.getFs().exists(new Path(metadataTablePath))) {
+          HoodieTableMetaClient mdtMetaClient = HoodieTableMetaClient.builder()
+              .setConf(metaClient.getHadoopConf()).setBasePath(metadataTablePath).build();
+          HoodieWriteConfig mdtWriteConfig = HoodieBackedTableMetadataWriter.createMetadataWriteConfig(
+              config, HoodieFailedWritesCleaningPolicy.LAZY);

Review Comment:
   why this has to be lazy? we can rollback any pending operations right ? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org