[jira] [Updated] (OOZIE-1167) Fix and rework PartitionDependency Management

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rohini Palaniswamy updated OOZIE-1167:
--------------------------------------

    Attachment: OOZIE-1167-2.patch

https://reviews.apache.org/r/8960/
                
> Fix and rework PartitionDependency Management
> ---------------------------------------------
>
>                 Key: OOZIE-1167
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1167
>             Project: Oozie
>          Issue Type: Sub-task
>            Reporter: Rohini Palaniswamy
>            Assignee: Rohini Palaniswamy
>         Attachments: OOZIE-1167-2.patch
>
>
> Currently the PDMService only works if the hcat dependency URI partitions match exactly with all the partitions in the notification. This is mostly not the case as there will be lot of sub-partitions in a table, but only the date partition will be specified for dependency check. So need to do a subset match. 
>   Also need to extract out the dependency cache and make an interface, so that a cache which spills to disk can later be substituted. Spillable cache is required for scale when number of pending dependencies will be high. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira