[jira] [Commented] (HUDI-525) inserts info miss in delta_commit_inflight meta file

["liujianhui (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17014094#comment-17014094 ] 

liujianhui commented on HUDI-525:
---------------------------------

{ code}

cat 20200113145851.deltacommit.inflight
{
 "partitionToWriteStats" : \{ },
 "compacted" : false,
 "extraMetadataMap" : \{ },
 "totalScanTime" : 0,
 "totalCreateTime" : 0,
 "totalUpsertTime" : 0,
 "totalCompactedRecordsUpdated" : 0,
 "totalLogFilesSize" : 0,
 "totalLogFilesCompacted" : 0,
 "fileIdAndRelativePaths" : \{ },
 "totalRecordsDeleted" : 0,
 "totalLogRecordsCompacted" : 0,
 "extraMetadata" : \{ }
}

{code}

the deltacommit should contains the insert info, event the field id is unknown at that moment

> inserts info miss in delta_commit_inflight meta file
> ----------------------------------------------------
>
>                 Key: HUDI-525
>                 URL: https://issues.apache.org/jira/browse/HUDI-525
>             Project: Apache Hudi (incubating)
>          Issue Type: Bug
>            Reporter: liujianhui
>            Priority: Major
>
> should add  insert info in WorkInfoStat
> {code}
> private void saveWorkloadProfileMetadataToInflight(WorkloadProfile profile, HoodieTable<T> table, String commitTime)
>  throws HoodieCommitException {
>  try {
>  HoodieCommitMetadata metadata = new HoodieCommitMetadata();
>  profile.getPartitionPaths().forEach(path -> {
>  WorkloadStat partitionStat = profile.getWorkloadStat(path.toString());
>  partitionStat.getUpdateLocationToCount().forEach((key, value) -> {
>  HoodieWriteStat writeStat = new HoodieWriteStat();
>  writeStat.setFileId(key);
>  // TODO : Write baseCommitTime is possible here ?
>  writeStat.setPrevCommit(value.getKey());
>  writeStat.setNumUpdateWrites(value.getValue());
>  metadata.addWriteStat(path.toString(), writeStat);
>  });
>  });
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)