You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "bvaradar (via GitHub)" <gi...@apache.org> on 2024/06/06 07:46:51 UTC

[PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

bvaradar opened a new pull request, #11406:
URL: https://github.com/apache/hudi/pull/11406

   [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table
   
   ### Change Logs
   
   [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table
   
   ### Impact
   
   Bump up table versions 
   
   ### Risk level (write none, low medium or high below)
   
   _none_
   ### Documentation Update
   
   _none_
   
   - _The config description must be updated if new configs are added or the default value of the configs are changed_
   - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
     ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
     changes to the website._
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11406:
URL: https://github.com/apache/hudi/pull/11406#discussion_r1630770670


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SixToSevenUpgradeHandler.java:
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.upgrade;
+
+import org.apache.hudi.common.config.ConfigProperty;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.config.HoodieWriteConfig;
+
+import java.util.Collections;
+import java.util.Map;
+
+/**
+ * Version 7 is going to be placeholder version for bridge release 0.16.0.
+ * Version 8 is the placeholder version to track 1.x.
+ */
+public class SixToSevenUpgradeHandler implements UpgradeHandler {
+  @Override
+  public Map<ConfigProperty, String> upgrade(HoodieWriteConfig config, HoodieEngineContext context,
+                                             String instantTime,
+                                             SupportsUpgradeDowngrade upgradeDowngradeHelper) {
+    return Collections.emptyMap();
+  }

Review Comment:
   When upgrading table from version 6 to 7, should we backfill the table initial version based on the existing table version?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions and introduce new hoodie table property to track initial table version [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2155749354

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24276",
       "triggerID" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5ff06d980691f473a959e149377e7aa14eaf7a55",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24287",
       "triggerID" : "5ff06d980691f473a959e149377e7aa14eaf7a55",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5ff06d9806",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5ff06d9806",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c UNKNOWN
   * 5ff06d980691f473a959e149377e7aa14eaf7a55 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24287) 
   * 5ff06d9806 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2151812964

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e8a80e29e51c84a3403906d2acf0aeee24dedda4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions and introduce new hoodie table property to track initial table version [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2155699424

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24276",
       "triggerID" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5ff06d980691f473a959e149377e7aa14eaf7a55",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24287",
       "triggerID" : "5ff06d980691f473a959e149377e7aa14eaf7a55",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c UNKNOWN
   * 5ff06d980691f473a959e149377e7aa14eaf7a55 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24287) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "balaji-varadarajan (via GitHub)" <gi...@apache.org>.
balaji-varadarajan commented on code in PR #11406:
URL: https://github.com/apache/hudi/pull/11406#discussion_r1631549810


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SixToSevenUpgradeHandler.java:
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.upgrade;
+
+import org.apache.hudi.common.config.ConfigProperty;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.config.HoodieWriteConfig;
+
+import java.util.Collections;
+import java.util.Map;
+
+/**
+ * Version 7 is going to be placeholder version for bridge release 0.16.0.
+ * Version 8 is the placeholder version to track 1.x.
+ */
+public class SixToSevenUpgradeHandler implements UpgradeHandler {
+  @Override
+  public Map<ConfigProperty, String> upgrade(HoodieWriteConfig config, HoodieEngineContext context,
+                                             String instantTime,
+                                             SupportsUpgradeDowngrade upgradeDowngradeHelper) {
+    return Collections.emptyMap();
+  }

Review Comment:
   We cannot determine the correct initial version during the upgrade path as we are doing one version increment at a time. We can basically interpret absence of INITIAL_VERSION as that the table was created by some version of 0.x 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions and introduce new hoodie table property to track initial table version [hudi]

Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua merged PR #11406:
URL: https://github.com/apache/hudi/pull/11406


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2155337800

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e8a80e29e51c84a3403906d2acf0aeee24dedda4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245) 
   * da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c UNKNOWN
   * 901c7f94b1b56ac19867d5d0deab34eb35ebce2c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "balaji-varadarajan (via GitHub)" <gi...@apache.org>.
balaji-varadarajan commented on code in PR #11406:
URL: https://github.com/apache/hudi/pull/11406#discussion_r1631546670


##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java:
##########
@@ -512,6 +519,15 @@ public HoodieTableVersion getTableVersion() {
         : VERSION.defaultValue();
   }
 
+  /**
+   * @return the hoodie.table.initial.version from hoodie.properties file.
+   */
+  public HoodieTableVersion getTableInitialVersion() {
+    return contains(INITIAL_VERSION)
+            ? HoodieTableVersion.versionFromCode(getInt(INITIAL_VERSION))

Review Comment:
   INITIAL_VERSION is similar to VERSION in type. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2151721570

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e8a80e29e51c84a3403906d2acf0aeee24dedda4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2155328082

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e8a80e29e51c84a3403906d2acf0aeee24dedda4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245) 
   * da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions and introduce new hoodie table property to track initial table version [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2155695586

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24276",
       "triggerID" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "5ff06d980691f473a959e149377e7aa14eaf7a55",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5ff06d980691f473a959e149377e7aa14eaf7a55",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c UNKNOWN
   * 901c7f94b1b56ac19867d5d0deab34eb35ebce2c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24276) 
   * 5ff06d980691f473a959e149377e7aa14eaf7a55 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11406:
URL: https://github.com/apache/hudi/pull/11406#discussion_r1629979770


##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -623,6 +624,8 @@ private static void initTableMetaClient(StorageConfiguration<?> storageConf, Str
     }
 
     initializeBootstrapDirsIfNotExists(basePath, storage);
+    // When the table is initialized, set the initial version to be the current version.
+    props.put(INITIAL_VERSION.key(), String.valueOf(HoodieTableVersion.current().versionCode()));

Review Comment:
   Are there no other callers of this method apart from `initTable`? It's important that the INITIAL_VERSION is not changed later due to offline/manual action right.



##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java:
##########
@@ -116,6 +116,13 @@ public class HoodieTableConfig extends HoodieConfig {
       .withDocumentation("Version of table, used for running upgrade/downgrade steps between releases with potentially "
           + "breaking/backwards compatible changes.");
 
+  public static final ConfigProperty<HoodieTableVersion> INITIAL_VERSION = ConfigProperty
+          .key("hoodie.table.initial.version")
+          .defaultValue(HoodieTableVersion.ZERO)
+          .withDocumentation("Initial Version of table when the table was created.Used for upgrade/downgrade"

Review Comment:
   ```suggestion
             .withDocumentation("Initial Version of table when the table was created. Used for upgrade/downgrade"
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions and introduce new hoodie table property to track initial table version [hudi]

Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2155725862

   Azure CI is green.
   <img width="1377" alt="Screenshot 2024-06-07 at 17 34 19" src="https://github.com/apache/hudi/assets/2497195/dfb773d1-1753-4bcd-9f05-27037985bf0a">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2155393533

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24245",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     }, {
       "hash" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24276",
       "triggerID" : "901c7f94b1b56ac19867d5d0deab34eb35ebce2c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * da08a0b3c0524b46e70a4cbed8ab82eb5f84f24c UNKNOWN
   * 901c7f94b1b56ac19867d5d0deab34eb35ebce2c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=24276) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11406:
URL: https://github.com/apache/hudi/pull/11406#issuecomment-2151706424

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e8a80e29e51c84a3403906d2acf0aeee24dedda4",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e8a80e29e51c84a3403906d2acf0aeee24dedda4 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "balaji-varadarajan (via GitHub)" <gi...@apache.org>.
balaji-varadarajan commented on code in PR #11406:
URL: https://github.com/apache/hudi/pull/11406#discussion_r1631564698


##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -623,6 +624,8 @@ private static void initTableMetaClient(StorageConfiguration<?> storageConf, Str
     }
 
     initializeBootstrapDirsIfNotExists(basePath, storage);
+    // When the table is initialized, set the initial version to be the current version.
+    props.put(INITIAL_VERSION.key(), String.valueOf(HoodieTableVersion.current().versionCode()));

Review Comment:
   Good point. Found one place in RepairsCommand and added the fix. 



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SixToSevenUpgradeHandler.java:
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.upgrade;
+
+import org.apache.hudi.common.config.ConfigProperty;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.config.HoodieWriteConfig;
+
+import java.util.Collections;
+import java.util.Map;
+
+/**
+ * Version 7 is going to be placeholder version for bridge release 0.16.0.
+ * Version 8 is the placeholder version to track 1.x.

Review Comment:
   Done



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SevenToSixDowngradeHandler.java:
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.upgrade;
+
+import org.apache.hudi.common.config.ConfigProperty;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.config.HoodieWriteConfig;
+
+import java.util.Collections;
+import java.util.Map;
+
+/**
+ * Version 7 is going to be placeholder version for bridge release 0.16.0.
+ * Version 8 is the placeholder version to track 1.x.

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11406:
URL: https://github.com/apache/hudi/pull/11406#discussion_r1631737898


##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SixToSevenUpgradeHandler.java:
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.upgrade;
+
+import org.apache.hudi.common.config.ConfigProperty;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.config.HoodieWriteConfig;
+
+import java.util.Collections;
+import java.util.Map;
+
+/**
+ * Version 7 is going to be placeholder version for bridge release 0.16.0.
+ * Version 8 is the placeholder version to track 1.x.
+ */
+public class SixToSevenUpgradeHandler implements UpgradeHandler {
+  @Override
+  public Map<ConfigProperty, String> upgrade(HoodieWriteConfig config, HoodieEngineContext context,
+                                             String instantTime,
+                                             SupportsUpgradeDowngrade upgradeDowngradeHelper) {
+    return Collections.emptyMap();
+  }

Review Comment:
   Makes sense.  Sounds good to me.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7834] Create placeholder table versions. Introduce new hoodie table property to track initial table version when table was created. This is needed to identify if the table was originally a 0.x or a 1.x table [hudi]

Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #11406:
URL: https://github.com/apache/hudi/pull/11406#discussion_r1630761262


##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java:
##########
@@ -512,6 +519,15 @@ public HoodieTableVersion getTableVersion() {
         : VERSION.defaultValue();
   }
 
+  /**
+   * @return the hoodie.table.initial.version from hoodie.properties file.
+   */
+  public HoodieTableVersion getTableInitialVersion() {
+    return contains(INITIAL_VERSION)
+            ? HoodieTableVersion.versionFromCode(getInt(INITIAL_VERSION))

Review Comment:
   Is the config value an integer or the String representing the `HoodieTableVersion` enum?



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SevenToSixDowngradeHandler.java:
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.upgrade;
+
+import org.apache.hudi.common.config.ConfigProperty;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.config.HoodieWriteConfig;
+
+import java.util.Collections;
+import java.util.Map;
+
+/**
+ * Version 7 is going to be placeholder version for bridge release 0.16.0.
+ * Version 8 is the placeholder version to track 1.x.

Review Comment:
   Nit: mention that there is no format change between table version 6 and 7.  Table version 7 is meant for required bridge release upgrade before upgrading to 1.0.



##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/SixToSevenUpgradeHandler.java:
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.upgrade;
+
+import org.apache.hudi.common.config.ConfigProperty;
+import org.apache.hudi.common.engine.HoodieEngineContext;
+import org.apache.hudi.config.HoodieWriteConfig;
+
+import java.util.Collections;
+import java.util.Map;
+
+/**
+ * Version 7 is going to be placeholder version for bridge release 0.16.0.
+ * Version 8 is the placeholder version to track 1.x.

Review Comment:
   Same here on docs.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org