[jira] [Commented] (OOZIE-1527) Fix scalability issues with coordinator materialization

["Purshotam Shah (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13999136#comment-13999136 ] 

Purshotam Shah commented on OOZIE-1527:
---------------------------------------

{quote}
Previously lookup interval was scheduling interval. If it was set to 2 mins and scheduling interval is not overriden (default 5 mins), then actions can materialize slowly. Need to make scheduling interval=min(lookup interval, scheduling interval). Reopening the jira to fix that.
{quote}
If scheduling interval is specified then it should fall back on lookup interval.

> Fix scalability issues with coordinator materialization
> -------------------------------------------------------
>
>                 Key: OOZIE-1527
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1527
>             Project: Oozie
>          Issue Type: Bug
>          Components: coordinator
>    Affects Versions: trunk
>            Reporter: Mona Chitnis
>            Assignee: Purshotam Shah
>             Fix For: trunk
>
>         Attachments: OOZIE-1527-V2.patch, OOZIE-1527-V3.patch
>
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> In certain situations when there is a large number of coordinators in the system, they have been observed to create huge backlog in materialization, and progressing very slow compared to expected. This patch can be looked upon as both a bug-fix or an enhancement addressing following points:
> 1. 'materialization.system.limit' leads to bringing Coord jobs in LRU fashion, but some of them may already be maxing out at actions to materialize (= throttle), and < limit jobs may actually undergo materialization. This patch does a second iteration of loading jobs to get materialized to reduce backlog
> 2. 'materialization.window' being 1 hour may work in most cases, but hourly jobs are seen to face significant slowdown at times, by lot of other minute jobs getting materialized. Therefore, window can be doubled (i.e. 2 hours) when job is hourly/daily.
> 3. For hourly coordinators, it is consistently seen that materialization occurs only near the end of the hour. e.g. for action whose nominal time is 2:00, action creation time is 1:59, if nominal time - 3:00, creation time is 2:58 and so on. If window is an hour in the future, doesn't explain why materialization won't occur anytime in the middle of the preceding hour.



--
This message was sent by Atlassian JIRA
(v6.2#6252)