You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by "Srikanth Sundarrajan (JIRA)" <ji...@apache.org> on 2013/07/18 16:56:48 UTC
[jira] [Updated] (FALCON-50) Falcon replication having issue to s3
[ https://issues.apache.org/jira/browse/FALCON-50?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Srikanth Sundarrajan updated FALCON-50:
---------------------------------------
Description:
Overview :- Replication of data from one cluster to s3 is not working.
Detail:-
Replication feed was submitted and scheduled through falcon. While only the retention bundle and co-ordinator (in oozie) are kicked up, could not see any sign of replication bundle or co-ordinator in oozie logs. Dont see any error in logs.
Below is the feed definition :-
?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="process-name" description="process-name" xmlns="uri:falcon:feed:0.1">
<frequency>hours(1)</frequency>
<late-arrival cut-off="hours(6)" />
<clusters>
<cluster name="cluster1" type="source">
<validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
<retention limit="days(24)" action="delete"/>
</cluster>
<cluster name="cluster2" type="target">
<validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
<retention limit="days(90)" action="delete"/>
<locations>
<location type="data" path="s3://<bucket-name>/<path-folder>/${YEAR}-${MONTH}-${DAY}-${HOUR}/"/>
</locations>
</cluster>
</clusters>
<locations>
<location type="data" path="/path/to/copy/${YEAR}/${MONTH}/${DAY}/${HOUR}/"/>
</locations>
<ACL owner="user" group="group" permission="0x755"/>
<schema location="/schema/s1" provider="protobuf"/>
<properties>
<property name="queueName" value="queueName"/>
<property name="jobPriority" value="NORMAL"/>
</properties>
</feed>
was:
Overview :- Replication of data from one cluster to s3 is not working.
Detail:-
Replication feed was submitted and scheduled through ivory. While only the retention bundle and co-ordinator (in oozie) are kicked up, could not see any sign of replication bundle or co-ordinator in oozie logs. Dont see any error in logs.
Below is the feed definition :-
?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed name="process-name" description="process-name" xmlns="uri:ivory:feed:0.1">
<frequency>hours(1)</frequency>
<late-arrival cut-off="hours(6)" />
<clusters>
<cluster name="cluster1" type="source">
<validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
<retention limit="days(24)" action="delete"/>
</cluster>
<cluster name="cluster2" type="target">
<validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
<retention limit="days(90)" action="delete"/>
<locations>
<location type="data" path="s4://<bucket-name>/<path-folder>/${YEAR}-${MONTH}-${DAY}-${HOUR}/"/>
</locations>
</cluster>
</clusters>
<locations>
<location type="data" path="/path/to/copy/${YEAR}/${MONTH}/${DAY}/${HOUR}/"/>
</locations>
<ACL owner="user" group="group" permission="0x755"/>
<schema location="/schema/s1" provider="protobuf"/>
<properties>
<property name="queueName" value="queueName"/>
<property name="jobPriority" value="NORMAL"/>
</properties>
</feed>
> Falcon replication having issue to s3
> -------------------------------------
>
> Key: FALCON-50
> URL: https://issues.apache.org/jira/browse/FALCON-50
> Project: Falcon
> Issue Type: Bug
> Components: general
> Reporter: Biswajit Nayak
> Labels: operability
>
> Overview :- Replication of data from one cluster to s3 is not working.
> Detail:-
> Replication feed was submitted and scheduled through falcon. While only the retention bundle and co-ordinator (in oozie) are kicked up, could not see any sign of replication bundle or co-ordinator in oozie logs. Dont see any error in logs.
> Below is the feed definition :-
> ?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <feed name="process-name" description="process-name" xmlns="uri:falcon:feed:0.1">
> <frequency>hours(1)</frequency>
> <late-arrival cut-off="hours(6)" />
> <clusters>
> <cluster name="cluster1" type="source">
> <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
> <retention limit="days(24)" action="delete"/>
> </cluster>
> <cluster name="cluster2" type="target">
> <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
> <retention limit="days(90)" action="delete"/>
> <locations>
> <location type="data" path="s3://<bucket-name>/<path-folder>/${YEAR}-${MONTH}-${DAY}-${HOUR}/"/>
> </locations>
> </cluster>
> </clusters>
> <locations>
> <location type="data" path="/path/to/copy/${YEAR}/${MONTH}/${DAY}/${HOUR}/"/>
> </locations>
> <ACL owner="user" group="group" permission="0x755"/>
> <schema location="/schema/s1" provider="protobuf"/>
> <properties>
> <property name="queueName" value="queueName"/>
> <property name="jobPriority" value="NORMAL"/>
> </properties>
> </feed>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira