You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by "Balu Vellanki (JIRA)" <ji...@apache.org> on 2015/09/03 02:28:46 UTC
[jira] [Comment Edited] (FALCON-1371) Status of scheduled Process
entity is shown as submitted in corner case
[ https://issues.apache.org/jira/browse/FALCON-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728280#comment-14728280 ]
Balu Vellanki edited comment on FALCON-1371 at 9/3/15 12:28 AM:
----------------------------------------------------------------
This patch handles the following scenarios.
1. When a bundle completed running successfully, status is now shown as "Succeeded" instead of "Submitted"
2. When the following feed is submitted, a coordinator is created for replication on backupCluster, but no coordinator is created on primaryCluster for retention because the feed validity on primaryCluster is in the past. There was a bug in code which was causing feed status to be shown as "submitted" if any of the clusters have a MISSING bundle. After this fix, the status will be shown as running.
{code}
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="cleansedEmailFeed" description="Cleansed customer emails" xmlns="uri:falcon:feed:0.1">
<groups>churnAnalysisDataPipeline</groups>
<frequency>hours(1)</frequency>
<timezone>UTC</timezone>
<late-arrival cut-off="hours(4)"/>
<clusters>
<cluster name="primaryCluster" type="source">
<validity start="2015-07-22T01:00Z" end="2015-07-22T03:00Z"/>
<retention limit="hours(90)" action="delete"/>
<locations>
<location type="data" path="/user/ambari-qa/falcon/demo/primary/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
<location type="stats" path="/"/>
<location type="meta" path="/"/>
</locations>
</cluster>
<cluster name="backupCluster" type="target">
<validity start="2015-07-22T01:00Z" end="2015-07-22T03:00Z"/>
<retention limit="hours(90)" action="delete"/>
<locations>
<location type="data" path="/falcon/demo/bcp/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
<location type="stats" path="/"/>
<location type="meta" path="/"/>
</locations>
</cluster>
</clusters>
<locations>
<location type="data" path="/user/ambari-qa/falcon/demo/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
<location type="stats" path="/"/>
<location type="meta" path="/"/>
</locations>
<ACL owner="ambari-qa" group="users" permission="0x755"/>
<schema location="/none" provider="/none"/>
</feed>
{code}
3. If a bundle is killed using oozie command line, the status of entity will be now shown as KILLED
4. Added support for FAILED.
5. Added methods to avoid calling findLatestBundle(...) repeatedly.
was (Author: bvellanki):
This patch handles the following scenarios.
1. When a bundle completed running successfully, status is now shown as "Succeeded" instead of "Submitted"
2. When the following feed is submitted, a coordinator is created for replication on backupCluster, but no coordinator is created on primaryCluster for retention because the feed validity on primaryCluster is in the past. There was a bug in code which was causing feed status to be shown as "submitted" if any of the clusters have a MISSING bundle. After this fix, the status will be shown as running.
{code}
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="cleansedEmailFeed" description="Cleansed customer emails" xmlns="uri:falcon:feed:0.1">
<groups>churnAnalysisDataPipeline</groups>
<frequency>hours(1)</frequency>
<timezone>UTC</timezone>
<late-arrival cut-off="hours(4)"/>
<clusters>
<cluster name="primaryCluster" type="source">
<validity start="2015-07-22T01:00Z" end="2015-07-22T03:00Z"/>
<retention limit="hours(90)" action="delete"/>
<locations>
<location type="data" path="/user/ambari-qa/falcon/demo/primary/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
<location type="stats" path="/"/>
<location type="meta" path="/"/>
</locations>
</cluster>
<cluster name="backupCluster" type="target">
<validity start="2015-07-22T01:00Z" end="2015-07-22T03:00Z"/>
<retention limit="hours(90)" action="delete"/>
<locations>
<location type="data" path="/falcon/demo/bcp/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
<location type="stats" path="/"/>
<location type="meta" path="/"/>
</locations>
</cluster>
</clusters>
<locations>
<location type="data" path="/user/ambari-qa/falcon/demo/processed/enron/${YEAR}-${MONTH}-${DAY}-${HOUR}"/>
<location type="stats" path="/"/>
<location type="meta" path="/"/>
</locations>
<ACL owner="ambari-qa" group="users" permission="0x755"/>
<schema location="/none" provider="/none"/>
</feed>
{code}
3. If a bundle is killed using oozie command line, the status of entity will be now shown as KILLED
4. Added support for FAILED.
5. Added methods to avoid calling findLatestBundle(...) repeatedly.
> Status of scheduled Process entity is shown as submitted in corner case
> -----------------------------------------------------------------------
>
> Key: FALCON-1371
> URL: https://issues.apache.org/jira/browse/FALCON-1371
> Project: Falcon
> Issue Type: Bug
> Affects Versions: 0.6.1
> Reporter: Balu Vellanki
> Assignee: Balu Vellanki
> Attachments: FALCON-1371.patch
>
>
> Create a process entity with
> - frequency of 1 hour
> - validity start as "2012-07-01T00:00Z"
> - and validity end as "2012-07-01T01:02Z"
> Now list the process entity after one hour. The process coordinator status in Oozie is "succeeded". But Falcon shows the status as submitted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)