You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@falcon.apache.org by Narayan Periwal <na...@inmobi.com> on 2015/08/10 12:59:08 UTC

Review Request 37292: FALCON-1385: Rerun for pipelines

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37292/
-----------------------------------------------------------

Review request for Falcon.


Repository: falcon-git


Description
-------

In case of backlogs, or cluster issues, we need to rerun all failed instances in an entire pipeline. Currently falcon supports rerun only on one entity or instances of an entity. An ability to filter desired several entities or instances from several entities and rerun them in one command will add lot of value to falcon.
Users should be able to use the following criteria to filter the instances which need to be rerun.
1) start date and end date
2) name 
3) tag key=value
4) pipeline (valid only in case of processes)
5) Status (should accept multiple values for status)
Need to pay attention on partial success / recovery of such operations.


Diffs
-----

  client/src/main/java/org/apache/falcon/ResponseHelper.java 2df3e00 
  client/src/main/java/org/apache/falcon/cli/FalconCLI.java 148f789 
  client/src/main/java/org/apache/falcon/client/FalconClient.java 9649e10 
  client/src/main/java/org/apache/falcon/resource/BulkRerunResult.java PRE-CREATION 
  common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 07fafb5 
  oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 2f3dc6f 
  prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java 43461c8 
  prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java 248ee7b 
  prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 465a636 
  webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java cbc936e 

Diff: https://reviews.apache.org/r/37292/diff/


Testing
-------

Done


Thanks,

Narayan Periwal

Re: Review Request 37292: FALCON-1385: Rerun for pipelines

Posted by Narayan Periwal <na...@inmobi.com>.


> On Aug. 10, 2015, 11:02 a.m., sandeep samudrala wrote:
> > client/src/main/java/org/apache/falcon/cli/FalconCLI.java, line 326
> > <https://reviews.apache.org/r/37292/diff/1/?file=1036156#file1036156line326>
> >
> >     can end not be optional?

Start and End dates cannot be empty for Instance POST apis.


- Narayan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37292/#review94721
-----------------------------------------------------------


On Aug. 10, 2015, 10:59 a.m., Narayan Periwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37292/
> -----------------------------------------------------------
> 
> (Updated Aug. 10, 2015, 10:59 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> In case of backlogs, or cluster issues, we need to rerun all failed instances in an entire pipeline. Currently falcon supports rerun only on one entity or instances of an entity. An ability to filter desired several entities or instances from several entities and rerun them in one command will add lot of value to falcon.
> Users should be able to use the following criteria to filter the instances which need to be rerun.
> 1) start date and end date
> 2) name 
> 3) tag key=value
> 4) pipeline (valid only in case of processes)
> 5) Status (should accept multiple values for status)
> Need to pay attention on partial success / recovery of such operations.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/ResponseHelper.java 2df3e00 
>   client/src/main/java/org/apache/falcon/cli/FalconCLI.java 148f789 
>   client/src/main/java/org/apache/falcon/client/FalconClient.java 9649e10 
>   client/src/main/java/org/apache/falcon/resource/BulkRerunResult.java PRE-CREATION 
>   common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 07fafb5 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 2f3dc6f 
>   prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java 43461c8 
>   prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java 248ee7b 
>   prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 465a636 
>   webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java cbc936e 
> 
> Diff: https://reviews.apache.org/r/37292/diff/
> 
> 
> Testing
> -------
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

Re: Review Request 37292: FALCON-1385: Rerun for pipelines

Posted by sandeep samudrala <sa...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37292/#review94721
-----------------------------------------------------------



client/src/main/java/org/apache/falcon/cli/FalconCLI.java (line 326)
<https://reviews.apache.org/r/37292/#comment149339>

    can end not be optional?


- sandeep samudrala


On Aug. 10, 2015, 10:59 a.m., Narayan Periwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37292/
> -----------------------------------------------------------
> 
> (Updated Aug. 10, 2015, 10:59 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> In case of backlogs, or cluster issues, we need to rerun all failed instances in an entire pipeline. Currently falcon supports rerun only on one entity or instances of an entity. An ability to filter desired several entities or instances from several entities and rerun them in one command will add lot of value to falcon.
> Users should be able to use the following criteria to filter the instances which need to be rerun.
> 1) start date and end date
> 2) name 
> 3) tag key=value
> 4) pipeline (valid only in case of processes)
> 5) Status (should accept multiple values for status)
> Need to pay attention on partial success / recovery of such operations.
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/falcon/ResponseHelper.java 2df3e00 
>   client/src/main/java/org/apache/falcon/cli/FalconCLI.java 148f789 
>   client/src/main/java/org/apache/falcon/client/FalconClient.java 9649e10 
>   client/src/main/java/org/apache/falcon/resource/BulkRerunResult.java PRE-CREATION 
>   common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 07fafb5 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 2f3dc6f 
>   prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java 43461c8 
>   prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java 248ee7b 
>   prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 465a636 
>   webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java cbc936e 
> 
> Diff: https://reviews.apache.org/r/37292/diff/
> 
> 
> Testing
> -------
> 
> Done
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>