You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by Narayan Periwal <na...@inmobi.com> on 2015/09/30 07:29:28 UTC
Review Request 38882: FALCON-1494: In instance status list,
show all runs for instances when requested by user
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/
-----------------------------------------------------------
Review request for Falcon.
Bugs: FALCON-1495
https://issues.apache.org/jira/browse/FALCON-1495
Repository: falcon-git
Description
-------
Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
Diffs
-----
client/src/main/java/org/apache/falcon/cli/FalconCLI.java c914649
client/src/main/java/org/apache/falcon/client/FalconClient.java 981559b
common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 0b560bb
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 6660af1
prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6
prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 3280789
prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 96c99f0
unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b
webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
Diff: https://reviews.apache.org/r/38882/diff/
Testing
-------
Done.
Thanks,
Narayan Periwal
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by pavan kumar kolamuri <pa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/#review101875
-----------------------------------------------------------
client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java (line 48)
<https://reviews.apache.org/r/38882/#comment159375>
Why public ?
client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java (line 154)
<https://reviews.apache.org/r/38882/#comment159376>
Can you change the description and say something like all attempts for job ?
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 825)
<https://reviews.apache.org/r/38882/#comment160188>
Can you use props and fit this logic in performAction method only ? Is this method really required ?
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 849)
<https://reviews.apache.org/r/38882/#comment160117>
You can use getWorkflowInfo here
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 859)
<https://reviews.apache.org/r/38882/#comment160118>
Won't instanceList contain duplicate entries? Initially one workflow got added and later all workflow's added .
- pavan kumar kolamuri
On Oct. 6, 2015, 1:22 p.m., Narayan Periwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38882/
> -----------------------------------------------------------
>
> (Updated Oct. 6, 2015, 1:22 p.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1495
> https://issues.apache.org/jira/browse/FALCON-1495
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
>
> As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
>
> In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java 24be7f1
> client/src/main/java/org/apache/falcon/client/FalconClient.java 20f6447
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 0b560bb
> docs/src/site/twiki/FalconCLI.twiki 22003d3
> docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 6660af1
> prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6
> prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 63e0647
> prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 96c99f0
> unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b
> webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
>
> Diff: https://reviews.apache.org/r/38882/diff/
>
>
> Testing
> -------
>
> Done.
>
>
> Thanks,
>
> Narayan Periwal
>
>
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by Pallavi Rao <pa...@inmobi.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/#review112383
-----------------------------------------------------------
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 672)
<https://reviews.apache.org/r/38882/#comment172874>
Instead of creating another method with duplicated code, can you think of may be using the instance populated by performAction to perform further action?
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 679)
<https://reviews.apache.org/r/38882/#comment172875>
The ERROR is set for the first instance in the list, although something down the line could have failed.
- Pallavi Rao
On Dec. 31, 2015, 5:23 a.m., Narayan Periwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38882/
> -----------------------------------------------------------
>
> (Updated Dec. 31, 2015, 5:23 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1495
> https://issues.apache.org/jira/browse/FALCON-1495
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
>
> As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
>
> In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java bfb1d65
> client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java e59af24
> client/src/main/java/org/apache/falcon/client/FalconClient.java 4a773c6
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java b53efe6
> docs/src/site/twiki/FalconCLI.twiki 26e6b33
> docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java b486357
> prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java d50d111
> prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 88131f3
> prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 801bd10
> scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java bceab72
> unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 13375ef
> unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 148cbf7
> unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 382e0c9
> webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
> webapp/src/test/java/org/apache/falcon/resource/InstanceSchedulerManagerJerseyIT.java 1523b76
> webapp/src/test/java/org/apache/falcon/resource/ProcessInstanceManagerIT.java 755efa3
>
> Diff: https://reviews.apache.org/r/38882/diff/
>
>
> Testing
> -------
>
> Done.
>
>
> Thanks,
>
> Narayan Periwal
>
>
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by Ajay Yadava <aj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/#review112606
-----------------------------------------------------------
client/src/main/java/org/apache/falcon/client/FalconClient.java (line 495)
<https://reviews.apache.org/r/38882/#comment173111>
Instead of using sendInstanceRequest method and making changes to all other API calls as well, it is better to create another method and use that for this change.
docs/src/site/twiki/FalconCLI.twiki (line 285)
<https://reviews.apache.org/r/38882/#comment173112>
This should go to cli/ListInstance.twiki
docs/src/site/twiki/FalconCLI.twiki (line 291)
<https://reviews.apache.org/r/38882/#comment173113>
Please attach an example with this option and result (in ListInstance.twiki)
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 685)
<https://reviews.apache.org/r/38882/#comment173115>
This condition can be moved above try and I believe then we can avoid repeating it 3 times.
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 847)
<https://reviews.apache.org/r/38882/#comment173114>
Instead of accepting a list and updating it, return the list as a result of this method.
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 855)
<https://reviews.apache.org/r/38882/#comment173243>
Use StringUtils.isNotBlank
scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java (line 343)
<https://reviews.apache.org/r/38882/#comment173245>
Please remove todo and raise a JIRA if required.
- Ajay Yadava
On Dec. 31, 2015, 5:23 a.m., Narayan Periwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38882/
> -----------------------------------------------------------
>
> (Updated Dec. 31, 2015, 5:23 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1495
> https://issues.apache.org/jira/browse/FALCON-1495
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
>
> As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
>
> In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java bfb1d65
> client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java e59af24
> client/src/main/java/org/apache/falcon/client/FalconClient.java 4a773c6
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java b53efe6
> docs/src/site/twiki/FalconCLI.twiki 26e6b33
> docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java b486357
> prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java d50d111
> prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 88131f3
> prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 801bd10
> scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java bceab72
> unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 13375ef
> unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 148cbf7
> unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 382e0c9
> webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
> webapp/src/test/java/org/apache/falcon/resource/InstanceSchedulerManagerJerseyIT.java 1523b76
> webapp/src/test/java/org/apache/falcon/resource/ProcessInstanceManagerIT.java 755efa3
>
> Diff: https://reviews.apache.org/r/38882/diff/
>
>
> Testing
> -------
>
> Done.
>
>
> Thanks,
>
> Narayan Periwal
>
>
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by sandeep samudrala <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/#review112376
-----------------------------------------------------------
Ship it!
Ship It!
- sandeep samudrala
On Dec. 31, 2015, 5:23 a.m., Narayan Periwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38882/
> -----------------------------------------------------------
>
> (Updated Dec. 31, 2015, 5:23 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1495
> https://issues.apache.org/jira/browse/FALCON-1495
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
>
> As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
>
> In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java bfb1d65
> client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java e59af24
> client/src/main/java/org/apache/falcon/client/FalconClient.java 4a773c6
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java b53efe6
> docs/src/site/twiki/FalconCLI.twiki 26e6b33
> docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java b486357
> prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java d50d111
> prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 88131f3
> prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 801bd10
> scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java bceab72
> unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 13375ef
> unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 148cbf7
> unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 382e0c9
> webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
> webapp/src/test/java/org/apache/falcon/resource/InstanceSchedulerManagerJerseyIT.java 1523b76
> webapp/src/test/java/org/apache/falcon/resource/ProcessInstanceManagerIT.java 755efa3
>
> Diff: https://reviews.apache.org/r/38882/diff/
>
>
> Testing
> -------
>
> Done.
>
>
> Thanks,
>
> Narayan Periwal
>
>
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by Pallavi Rao <pa...@inmobi.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/#review113031
-----------------------------------------------------------
Ship it!
Good to go once the minor refactor pointed out is addressed.
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 691)
<https://reviews.apache.org/r/38882/#comment173523>
One try-catch block should suffice with the if-else inside the try.
- Pallavi Rao
On Jan. 6, 2016, 7:22 a.m., Narayan Periwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38882/
> -----------------------------------------------------------
>
> (Updated Jan. 6, 2016, 7:22 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1495
> https://issues.apache.org/jira/browse/FALCON-1495
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
>
> As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
>
> In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java bfb1d65
> client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java e59af24
> client/src/main/java/org/apache/falcon/client/FalconClient.java 4a773c6
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java b53efe6
> docs/src/site/twiki/FalconCLI.twiki 26e6b33
> docs/src/site/twiki/falconcli/ListInstance.twiki 28e9fed
> docs/src/site/twiki/falconcli/StatusInstance.twiki f0392b3
> docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
> docs/src/site/twiki/restapi/InstanceStatus.twiki 53f512f
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java b486357
> prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java d50d111
> prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 88131f3
> prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 801bd10
> scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java bceab72
> unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 13375ef
> unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 148cbf7
> unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 382e0c9
> webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
> webapp/src/test/java/org/apache/falcon/resource/InstanceSchedulerManagerJerseyIT.java 1523b76
> webapp/src/test/java/org/apache/falcon/resource/ProcessInstanceManagerIT.java 755efa3
>
> Diff: https://reviews.apache.org/r/38882/diff/
>
>
> Testing
> -------
>
> Done.
>
>
> Thanks,
>
> Narayan Periwal
>
>
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by Ajay Yadava <aj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/#review113726
-----------------------------------------------------------
Ship it!
Ship It!
docs/src/site/twiki/restapi/InstanceList.twiki (line 129)
<https://reviews.apache.org/r/38882/#comment174545>
ultra minor nit: Run ID should be increased. Will fix it while committing it.
- Ajay Yadava
On Jan. 6, 2016, 7:22 a.m., Narayan Periwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38882/
> -----------------------------------------------------------
>
> (Updated Jan. 6, 2016, 7:22 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1495
> https://issues.apache.org/jira/browse/FALCON-1495
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
>
> As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
>
> In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java bfb1d65
> client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java e59af24
> client/src/main/java/org/apache/falcon/client/FalconClient.java 4a773c6
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java b53efe6
> docs/src/site/twiki/FalconCLI.twiki 26e6b33
> docs/src/site/twiki/falconcli/ListInstance.twiki 28e9fed
> docs/src/site/twiki/falconcli/StatusInstance.twiki f0392b3
> docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
> docs/src/site/twiki/restapi/InstanceStatus.twiki 53f512f
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java b486357
> prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java d50d111
> prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 88131f3
> prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 801bd10
> scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java bceab72
> unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 13375ef
> unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 148cbf7
> unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 382e0c9
> webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
> webapp/src/test/java/org/apache/falcon/resource/InstanceSchedulerManagerJerseyIT.java 1523b76
> webapp/src/test/java/org/apache/falcon/resource/ProcessInstanceManagerIT.java 755efa3
>
> Diff: https://reviews.apache.org/r/38882/diff/
>
>
> Testing
> -------
>
> Done.
>
>
> Thanks,
>
> Narayan Periwal
>
>
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by Narayan Periwal <na...@inmobi.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/
-----------------------------------------------------------
(Updated Jan. 6, 2016, 7:22 a.m.)
Review request for Falcon.
Bugs: FALCON-1495
https://issues.apache.org/jira/browse/FALCON-1495
Repository: falcon-git
Description
-------
Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
Diffs (updated)
-----
client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java bfb1d65
client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java e59af24
client/src/main/java/org/apache/falcon/client/FalconClient.java 4a773c6
common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java b53efe6
docs/src/site/twiki/FalconCLI.twiki 26e6b33
docs/src/site/twiki/falconcli/ListInstance.twiki 28e9fed
docs/src/site/twiki/falconcli/StatusInstance.twiki f0392b3
docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
docs/src/site/twiki/restapi/InstanceStatus.twiki 53f512f
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java b486357
prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java d50d111
prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 88131f3
prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 801bd10
scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java bceab72
unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 13375ef
unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 148cbf7
unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 382e0c9
webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
webapp/src/test/java/org/apache/falcon/resource/InstanceSchedulerManagerJerseyIT.java 1523b76
webapp/src/test/java/org/apache/falcon/resource/ProcessInstanceManagerIT.java 755efa3
Diff: https://reviews.apache.org/r/38882/diff/
Testing
-------
Done.
Thanks,
Narayan Periwal
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by Narayan Periwal <na...@inmobi.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/
-----------------------------------------------------------
(Updated Dec. 31, 2015, 5:23 a.m.)
Review request for Falcon.
Bugs: FALCON-1495
https://issues.apache.org/jira/browse/FALCON-1495
Repository: falcon-git
Description
-------
Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
Diffs (updated)
-----
client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java bfb1d65
client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java e59af24
client/src/main/java/org/apache/falcon/client/FalconClient.java 4a773c6
common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java b53efe6
docs/src/site/twiki/FalconCLI.twiki 26e6b33
docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java b486357
prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java d50d111
prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 88131f3
prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 801bd10
scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java bceab72
unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 13375ef
unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 148cbf7
unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java 382e0c9
webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
webapp/src/test/java/org/apache/falcon/resource/InstanceSchedulerManagerJerseyIT.java 1523b76
webapp/src/test/java/org/apache/falcon/resource/ProcessInstanceManagerIT.java 755efa3
Diff: https://reviews.apache.org/r/38882/diff/
Testing
-------
Done.
Thanks,
Narayan Periwal
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by sandeep samudrala <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/#review110634
-----------------------------------------------------------
client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java (line 155)
<https://reviews.apache.org/r/38882/#comment170635>
to get all attempts of corresponding instances.
docs/src/site/twiki/FalconCLI.twiki (line 282)
<https://reviews.apache.org/r/38882/#comment170639>
end user doesn't understant run-id.
Change it to
"To get all the attempts for corresponding instances."
docs/src/site/twiki/FalconCLI.twiki (line 283)
<https://reviews.apache.org/r/38882/#comment170638>
Add a line over here.
docs/src/site/twiki/restapi/InstanceList.twiki (line 28)
<https://reviews.apache.org/r/38882/#comment170640>
same here.
As now with oozie not creating a new workflow whil e normal falcon retry or rerun, the attempts are gettable from oozie only when there are force reruns.
Document accordingly to let user understand that the attempts of full forced runs are only available.
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java (line 678)
<https://reviews.apache.org/r/38882/#comment170648>
instance1 is not used anywhere else?
- sandeep samudrala
On Oct. 27, 2015, 9:54 a.m., Narayan Periwal wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38882/
> -----------------------------------------------------------
>
> (Updated Oct. 27, 2015, 9:54 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-1495
> https://issues.apache.org/jira/browse/FALCON-1495
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
>
> As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
>
> In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java 6ea34da
> client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 91d5324
> client/src/main/java/org/apache/falcon/client/FalconClient.java 27510f6
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 8b3460a
> docs/src/site/twiki/FalconCLI.twiki d994866
> docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 09c29ab
> prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java 606f741
> prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java d317aa1
> prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 96c99f0
> scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java 8dcf3a5
> unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java b5afae3
> unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 1503b28
> unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java ac478f4
> webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
>
> Diff: https://reviews.apache.org/r/38882/diff/
>
>
> Testing
> -------
>
> Done.
>
>
> Thanks,
>
> Narayan Periwal
>
>
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by Narayan Periwal <na...@inmobi.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/
-----------------------------------------------------------
(Updated Oct. 27, 2015, 9:54 a.m.)
Review request for Falcon.
Bugs: FALCON-1495
https://issues.apache.org/jira/browse/FALCON-1495
Repository: falcon-git
Description
-------
Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
Diffs (updated)
-----
client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java 6ea34da
client/src/main/java/org/apache/falcon/client/AbstractFalconClient.java 91d5324
client/src/main/java/org/apache/falcon/client/FalconClient.java 27510f6
common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 8b3460a
docs/src/site/twiki/FalconCLI.twiki d994866
docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 09c29ab
prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java 606f741
prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java d317aa1
prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 96c99f0
scheduler/src/main/java/org/apache/falcon/workflow/engine/FalconWorkflowEngine.java 8dcf3a5
unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java b5afae3
unit/src/main/java/org/apache/falcon/unit/LocalInstanceManager.java 1503b28
unit/src/test/java/org/apache/falcon/unit/FalconUnitTestBase.java ac478f4
webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
Diff: https://reviews.apache.org/r/38882/diff/
Testing
-------
Done.
Thanks,
Narayan Periwal
Re: Review Request 38882: FALCON-1495: In instance status list,
show all runs for instances when requested by user
Posted by Narayan Periwal <na...@inmobi.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38882/
-----------------------------------------------------------
(Updated Oct. 6, 2015, 1:22 p.m.)
Review request for Falcon.
Summary (updated)
-----------------
FALCON-1495: In instance status list, show all runs for instances when requested by user
Bugs: FALCON-1495
https://issues.apache.org/jira/browse/FALCON-1495
Repository: falcon-git
Description
-------
Most of the times , when there are failures in the pipelines, retries run and the workflows might have got succeeded in sub sequential retry. But for end user to figure out why it has failed initially might be useful to understand for failures handling and SLA misses.
As of now to figure out the same, its a tedious task to get such information by either going through oozie DB/logs or via the logs backed up in hdfs by the falcon postprocessing(which is 100% reliable as in few cases, falcon post processing might fail or for some reason if it couldn't back up the logs).
In either case, Falcon should abstract the oozie client layer to be used by end user, and support all runs status to end user.
Diffs (updated)
-----
client/src/main/java/org/apache/falcon/cli/FalconInstanceCLI.java 24be7f1
client/src/main/java/org/apache/falcon/client/FalconClient.java 20f6447
common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java 0b560bb
docs/src/site/twiki/FalconCLI.twiki 22003d3
docs/src/site/twiki/restapi/InstanceList.twiki 229d6f9
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 6660af1
prism/src/main/java/org/apache/falcon/resource/AbstractInstanceManager.java c1b4da6
prism/src/main/java/org/apache/falcon/resource/AbstractSchedulableEntityManager.java 63e0647
prism/src/main/java/org/apache/falcon/resource/proxy/InstanceManagerProxy.java 96c99f0
unit/src/main/java/org/apache/falcon/unit/FalconUnitClient.java 169614b
webapp/src/main/java/org/apache/falcon/resource/InstanceManager.java 76c66f3
Diff: https://reviews.apache.org/r/38882/diff/
Testing
-------
Done.
Thanks,
Narayan Periwal