[jira] [Created] (HUDI-3975) Checksum can be wrong after table upgrade from version 3 to 4

["Ethan Guo (Jira)" <ji...@apache.org>]

Ethan Guo created HUDI-3975:
-------------------------------

             Summary: Checksum can be wrong after table upgrade from version 3 to 4
                 Key: HUDI-3975
                 URL: https://issues.apache.org/jira/browse/HUDI-3975
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: Ethan Guo
            Assignee: Ethan Guo
             Fix For: 0.11.0


In ThreeToFourUpgradeHandler, more table properties can be added after checksum is generated, causing the checksum to be wrong.
{code:java}
@Override
public Map<ConfigProperty, String> upgrade(HoodieWriteConfig config, HoodieEngineContext context, String instantTime, SupportsUpgradeDowngrade upgradeDowngradeHelper) {
  Map<ConfigProperty, String> tablePropsToAdd = new Hashtable<>();
  tablePropsToAdd.put(TABLE_CHECKSUM, String.valueOf(HoodieTableConfig.generateChecksum(config.getProps())));
  // if metadata is enabled and files partition exist then update TABLE_METADATA_INDEX_COMPLETED
  // schema for the files partition is same between the two versions
  if (config.isMetadataTableEnabled() && metadataPartitionExists(config.getBasePath(), context, MetadataPartitionType.FILES)) {
    tablePropsToAdd.put(TABLE_METADATA_PARTITIONS, MetadataPartitionType.FILES.getPartitionPath());
  }
  return tablePropsToAdd;
} {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)