[GitHub] [hudi] RocMarshal edited a comment on pull request #3813: [HUDI-2563][hudi-client] Refactor CompactionTriggerStrategy.

[GitBox <gi...@apache.org>]

RocMarshal edited a comment on pull request #3813:
URL: https://github.com/apache/hudi/pull/3813#issuecomment-968271847


   @xushiyan @yanghua @leesf .
   Excuse me.I looked into this CI failure case in the backgroud of this pr. There are some interesting appearance in test cases , such as `TestHoodieSparkMergeOnReadTableInsertUpdateDelete` referenced in the CI result. 
   
   `      
         
        String newCommitTime = "001"; // ---------------(1)
   
         client.startCommitWithTime(newCommitTime);
   
         List<HoodieRecord> records = dataGen.generateInserts(newCommitTime, 200);
   
         insertRecords(metaClient, records, client, cfg, newCommitTime);
   `
   
   The new `CompactionTriggerStrategy` will use commit instant time parameters and use delta commit info parameters to judge the state. 
   When the variable "001" tagged with `(1)` was transported into `HoodieInstantTimeGenerator#parseInstantTime(String timestamp)` , the `newCommitTime` value was not compatible with the parse pattern `yyyyMMddHHmmss` declared in `HoodieInstantTimeGenerator#INSTANT_TIME_FORMATTER`. At the same time, the compact configuration item was set as 'NUM_COMMITS' . As a result of the setting, the old `CompactionTriggerStrategy` would judge the state without instantTime parameters. The new `CompactionTriggerStrategy` will use both instantTime and commitNumber parameters to judge the state  on the contrary, which is the reason why the old  `CompactionTriggerStrategy` could be successful but new `CompactionTriggerStrategy` would be unsuccessful. With my limited read, we could enhance the validation of commitTime `(1)` in the target format `yyyyMMddHHmmss` or stop the refactor in the pr. 
   
   Please let me know what's your opinion.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org