You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Aditya Sharma (JIRA)" <ji...@apache.org> on 2017/09/26 16:33:00 UTC

[jira] [Created] (GOBBLIN-267) HiveSource creates workunit even when update time is before maxLookBackDays

Aditya Sharma created GOBBLIN-267:
-------------------------------------

             Summary: HiveSource creates workunit even when update time is before maxLookBackDays
                 Key: GOBBLIN-267
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-267
             Project: Apache Gobblin
          Issue Type: Bug
          Components: misc
            Reporter: Aditya Sharma
            Assignee: Aditya Sharma


org.apache.gobblin.data.management.conversion.hive.source.HiveSource creates workunit if:
1) Create time is after maxLookBackDays
2) Update time is greater than watermark

Since there are multiple policies to decide update time, it can happen that create time is greater than update time and hence maxLookBackDays will be redundant.

HiveSource should check for maxLookBackDays corresponding to update time



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)