[jira] [Commented] (OOZIE-1527) Fix scalability issues with coordinator materialization

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768682#comment-13768682 ] 

Rohini Palaniswamy commented on OOZIE-1527:
-------------------------------------------

We don't want to do 2 as it will produce a lot of coordinators which will wait on input. What we need to fix is the calculation of next materialization time in case of catchup jobs to make catchup hourly jobs run faster in case of huge number of coordinators in the system.
                
> Fix scalability issues with coordinator materialization
> -------------------------------------------------------
>
>                 Key: OOZIE-1527
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1527
>             Project: Oozie
>          Issue Type: Bug
>          Components: coordinator
>    Affects Versions: trunk
>            Reporter: Mona Chitnis
>            Assignee: Mona Chitnis
>             Fix For: trunk
>
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> In certain situations when there is a large number of coordinators in the system, they have been observed to create huge backlog in materialization, and progressing very slow compared to expected. This patch can be looked upon as both a bug-fix or an enhancement addressing following points:
> 1. 'materialization.system.limit' leads to bringing Coord jobs in LRU fashion, but some of them may already be maxing out at actions to materialize (= throttle), and < limit jobs may actually undergo materialization. This patch does a second iteration of loading jobs to get materialized to reduce backlog
> 2. 'materialization.window' being 1 hour may work in most cases, but hourly jobs are seen to face significant slowdown at times, by lot of other minute jobs getting materialized. Therefore, window can be doubled (i.e. 2 hours) when job is hourly/daily.
> 3. For hourly coordinators, it is consistently seen that materialization occurs only near the end of the hour. e.g. for action whose nominal time is 2:00, action creation time is 1:59, if nominal time - 3:00, creation time is 2:58 and so on. If window is an hour in the future, doesn't explain why materialization won't occur anytime in the middle of the preceding hour.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira