Re: Review Request 17991: OOZIE-1532 Purging should remove completed children job for long running coordinator jobs

[Bowen Zhang <bo...@yahoo.com>]

> On March 10, 2014, 3:09 a.m., Rohini Palaniswamy wrote:
> > The approach is inefficient. Getting the list of coord actions and deleting one by one will create big redo logs and also will be time consuming. 
> > Can you please do a join query to do the delete directly. It will simple, efficient in terms of DB load and lot less code as well. Something like
> > 
> > delete from CoordinatorActionBean a where a.id in (select w.parentId from WorkflowJobBean w where w.endTimestamp < :endTime and w.parentId is not null)
> > 
> > 
> > Can you also make the new query part of CoordActionQueryExecutor? We are not writing any new JPAExecutor classes.
> 
> Rohini Palaniswamy wrote:
>     Since we are not relying on something like ON DELETE CASCADE, the workflows need to be purged before coord actions can be purged. So w.parentId like '%C@%' is still needed in GET_COMPLETED_WORKFLOWS_WITH_NO_PARENT_OLDER_THAN (Can we rename it as it now includes workflows with parent as well?) so that the workflows are purged before coordinator is purged. In that case, after the workflow purge, the coord action purge query will become something like
>     
>     delete from CoordinatorActionBean a where a.externalId not in (select w.id from WorkflowJobBean w where w.parentId is not null)

Your approach of running everything in one query will change the existing mechanism of purging. Take a look at purgeXCommand.java and loadState and execute method.


- Bowen


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17991/#review36620
-----------------------------------------------------------


On Feb. 12, 2014, 5:05 a.m., Srikanth Sundarrajan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17991/
> -----------------------------------------------------------
> 
> (Updated Feb. 12, 2014, 5:05 a.m.)
> 
> 
> Review request for oozie.
> 
> 
> Bugs: OOZIE-1532
>     https://issues.apache.org/jira/browse/OOZIE-1532
> 
> 
> Repository: oozie-git
> 
> 
> Description
> -------
> 
> Purging should remove completed children job for long running coordinator jobs
> 
> 
> Diffs
> -----
> 
>   core/src/main/java/org/apache/oozie/CoordinatorActionBean.java 03a7ed8 
>   core/src/main/java/org/apache/oozie/WorkflowJobBean.java 3194995 
>   core/src/main/java/org/apache/oozie/command/PurgeXCommand.java 9973719 
>   core/src/main/java/org/apache/oozie/executor/jpa/CoordActionsDeleteJPAExecutor.java PRE-CREATION 
>   core/src/main/java/org/apache/oozie/executor/jpa/CoordActionsGetForPurgeJPAExecutor.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/command/TestPurgeXCommand.java 666271e 
>   core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionsDeleteJPAExecutor.java PRE-CREATION 
>   core/src/test/java/org/apache/oozie/executor/jpa/TestCoordActionsGetForPurgeJPAExecutor.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/17991/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Srikanth Sundarrajan
> 
>