You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Hung Tran (JIRA)" <ji...@apache.org> on 2017/09/27 04:23:00 UTC

[jira] [Resolved] (GOBBLIN-267) HiveSource creates workunit even when update time is before maxLookBackDays

     [ https://issues.apache.org/jira/browse/GOBBLIN-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hung Tran resolved GOBBLIN-267.
-------------------------------
    Resolution: Fixed

Issue resolved by pull request #2119
[https://github.com/apache/incubator-gobblin/pull/2119]

> HiveSource creates workunit even when update time is before maxLookBackDays
> ---------------------------------------------------------------------------
>
>                 Key: GOBBLIN-267
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-267
>             Project: Apache Gobblin
>          Issue Type: Bug
>          Components: misc
>            Reporter: Aditya Sharma
>            Assignee: Aditya Sharma
>
> org.apache.gobblin.data.management.conversion.hive.source.HiveSource creates workunit if:
> 1) Create time is after maxLookBackDays
> 2) Update time is greater than watermark
> Since there are multiple policies to decide update time, it can happen that create time is greater than update time and hence maxLookBackDays will be redundant.
> HiveSource should check for maxLookBackDays corresponding to update time



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)