You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Abhishek Tiwari (JIRA)" <ji...@apache.org> on 2018/01/03 06:42:01 UTC

[jira] [Updated] (GOBBLIN-267) HiveSource creates workunit even when update time is before maxLookBackDays

     [ https://issues.apache.org/jira/browse/GOBBLIN-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Abhishek Tiwari updated GOBBLIN-267:
------------------------------------
    Fix Version/s: 0.12

> HiveSource creates workunit even when update time is before maxLookBackDays
> ---------------------------------------------------------------------------
>
>                 Key: GOBBLIN-267
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-267
>             Project: Apache Gobblin
>          Issue Type: Bug
>          Components: misc
>            Reporter: Aditya Sharma
>            Assignee: Aditya Sharma
>             Fix For: 0.12
>
>
> org.apache.gobblin.data.management.conversion.hive.source.HiveSource creates workunit if:
> 1) Create time is after maxLookBackDays
> 2) Update time is greater than watermark
> Since there are multiple policies to decide update time, it can happen that create time is greater than update time and hence maxLookBackDays will be redundant.
> HiveSource should check for maxLookBackDays corresponding to update time



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)