You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Sagar Sumit (Jira)" <ji...@apache.org> on 2023/09/12 12:47:00 UTC
[jira] [Closed] (HUDI-6724) Initializing prevInstance to HoodieTimeline.INIT_INSTANT_TS to avoid partial reading of first commit
[ https://issues.apache.org/jira/browse/HUDI-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sagar Sumit closed HUDI-6724.
-----------------------------
Fix Version/s: 0.14.0
Resolution: Fixed
> Initializing prevInstance to HoodieTimeline.INIT_INSTANT_TS to avoid partial reading of first commit
> ----------------------------------------------------------------------------------------------------
>
> Key: HUDI-6724
> URL: https://issues.apache.org/jira/browse/HUDI-6724
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Lokesh Lingarajan
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.14.0
>
>
> Since object based incr jobs now have batching with in the commit, we can end-up in a situation for the first commit where prevInstance is same as startInstance according to existing code for batches within the first commit.
> In this scenario when we incremental query rows > prevInstance, we will skip the first commit as startInstance is also pointing to the same commit.
> This is due to defaulting prevInstance to startInstance in
> generateQueryInfo API.
> Fix is to have this default to HoodieTimeline.INIT_INSTANT_TS so batching can continue on the first commit
--
This message was sent by Atlassian Jira
(v8.20.10#820010)