[jira] [Commented] (OOZIE-1533) Coordinator action materialization is too slow due to coarse job level locks

["Shwetha G S (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13989476#comment-13989476 ] 

Shwetha G S commented on OOZIE-1533:
------------------------------------

[~rohini]
{quote}
CoordActionInputCheckXCommand might not be a good idea as it will put a lot of pressure on the namenode and also the Oozie database.
{quote}
Won't CallableQueueService.maxCallableConcurrency help here? The concurrency is on type which is same for all CoordActionInputCheckXCommand.

[~virag]
{quote}
 For e.g, if one job has 10,000 actions and other job has only 1 action and command queue size is 10,000. With coord job lock, only one command of any coordinator job can reside in the queue.
{quote}
The de-duping for queue is on action id for CoordActionInputCheckXCommand. So, 10,000 actions will be in the queue even with coord job lock.


> Coordinator action materialization is too slow due to coarse job level locks
> ----------------------------------------------------------------------------
>
>                 Key: OOZIE-1533
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1533
>             Project: Oozie
>          Issue Type: Improvement
>            Reporter: Srikanth Sundarrajan
>            Assignee: Srikanth Sundarrajan
>              Labels: locking
>         Attachments: OOZIE-1533.patch
>
>
> Coord job level lock introduces high contention. Instead introduce coord action level locking whenever appropriate



--
This message was sent by Atlassian JIRA
(v6.2#6252)