You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Jaydeep Vishwakarma (JIRA)" <ji...@apache.org> on 2015/04/24 12:47:38 UTC
[jira] [Commented] (OOZIE-2216) Aperiodic Data handling in oozie
[ https://issues.apache.org/jira/browse/OOZIE-2216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510793#comment-14510793 ]
Jaydeep Vishwakarma commented on OOZIE-2216:
--------------------------------------------
I have put my thoughts on the document. I think this feature will give oozie a new dimension. Please have a look and provide your valuable comments.
> Aperiodic Data handling in oozie
> --------------------------------
>
> Key: OOZIE-2216
> URL: https://issues.apache.org/jira/browse/OOZIE-2216
> Project: Oozie
> Issue Type: New Feature
> Components: coordinator
> Reporter: Jaydeep Vishwakarma
> Assignee: Jaydeep Vishwakarma
> Attachments: Oozie_aperiodic_data_handling.pdf
>
>
> Currently Oozie scheduling works on periodic datasets. It does not have any mechanism to handle aperiodic datasets, which doesn’t follow a fixed schedule/frequency.
> Use cases
> When incoming dataset arrives with no fixed schedule.
> Need to trigger the job based all data available since last run with a possible cap on the max size to process in one run.
> Try to avoid creating so many instances when you know input instances will be very few.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)