[jira] [Commented] (FALCON-168) Different validity of clusters in feed not handled

["Venkatesh Seetharam (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FALCON-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13814362#comment-13814362 ] 

Venkatesh Seetharam commented on FALCON-168:
--------------------------------------------

I think I found the offending code I have introduced with copy and paste error:
org.apache.falcon.converter.OozieFeedMapper.ReplicationOozieWorkflowMapper#createAndGetCoord

{code}
            Date sourceStartDate = getStartDate(feed, srcCluster, replicationDelayInMillis);
            Date sourceEndDate = getEndDate(feed, srcCluster);

            Date targetStartDate = getStartDate(feed, srcCluster, replicationDelayInMillis);
            Date targetEndDate = getEndDate(feed, srcCluster);
{code}
Notice the use of srcCluster as argument for both source and target. This should be:
{code}
            Date sourceStartDate = getStartDate(feed, srcCluster, replicationDelayInMillis);
            Date sourceEndDate = getEndDate(feed, srcCluster);

            Date targetStartDate = getStartDate(feed, trgCluster, replicationDelayInMillis);
            Date targetEndDate = getEndDate(feed, trgCluster);
{code}
Have added a few unit tests to cover this. Will have a patch in a bit.

> Different validity of clusters in feed not handled 
> ---------------------------------------------------
>
>                 Key: FALCON-168
>                 URL: https://issues.apache.org/jira/browse/FALCON-168
>             Project: Falcon
>          Issue Type: Bug
>          Components: replication
>         Environment: QA
>            Reporter: Samarth Gupta
>            Assignee: Venkatesh Seetharam
>
> Feed.xml has option to provide different validity for same feed on different clusters. When such a feed is scheduled the validity of source and target must be considered and overlapping time duration should be replicated. 
> Following test case fails for the patch available in FALCON-163
> 1. source validity : start="2012-10-01T12:00Z end="2099-01-01T00:00Z" 
> 2. target one validity : start="2012-10-01T12:05Z" end="2012-10-01T12:11Z"
> 3. target two validity : start="2012-10-01T12:10Z" end="2012-10-01T12:26Z"
> What should happen :
> based on the above, when is feed is scheduled replication start should be from:
> 1. target one : start="2012-10-01T12:05Z" and  end="2012-10-01T12:11Z"
> 2. target two:  start="2012-10-01T12:10Z" and end="2012-10-01T12:26Z"
> but ... What is happening after FALCON-163 patch:
> Both target one and two replication : start="2012-10-01T12:00Z" and  end="2099-01-01T00:00Z"
> Due to this instance are starting on target cluster when feed is not valid and such replication is failing with error "variable [distcpTargetPaths] cannot be resolved"
> feed refer to feed available in FALCON-95 for details. 



--
This message was sent by Atlassian JIRA
(v6.1#6144)