You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Srikanth Sundarrajan <sr...@hotmail.com> on 2015/10/26 08:28:07 UTC
Review Request 39641: PIG-4698 Enable dynamic resource
allocation/de-allocation on Yarn backends
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39641/
-----------------------------------------------------------
Review request for pig.
Bugs: PIG-4698
https://issues.apache.org/jira/browse/PIG-4698
Repository: pig-git
Description
-------
Resource elasticity needs to be enabled on Yarn backend to allow jobs to scale out better and provide better wall clock execution times, while unused resources should be released back to RM for use.
Diffs
-----
src/docs/src/documentation/content/xdocs/start.xml eedd5b7
src/org/apache/pig/backend/hadoop/executionengine/spark/SparkLauncher.java b542013
Diff: https://reviews.apache.org/r/39641/diff/
Testing
-------
Verified that the dynamic configuration is hornoured by the yarn system. Requires the auxillary shuffle service need to be enabled at the node manager and application level for this to work correctly.
Thanks,
Srikanth Sundarrajan
Re: Review Request 39641: PIG-4698 Enable dynamic resource
allocation/de-allocation on Yarn backends
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39641/#review104134
-----------------------------------------------------------
Ship it!
Ship It!
- Xuefu Zhang
On Oct. 26, 2015, 7:28 a.m., Srikanth Sundarrajan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39641/
> -----------------------------------------------------------
>
> (Updated Oct. 26, 2015, 7:28 a.m.)
>
>
> Review request for pig.
>
>
> Bugs: PIG-4698
> https://issues.apache.org/jira/browse/PIG-4698
>
>
> Repository: pig-git
>
>
> Description
> -------
>
> Resource elasticity needs to be enabled on Yarn backend to allow jobs to scale out better and provide better wall clock execution times, while unused resources should be released back to RM for use.
>
>
> Diffs
> -----
>
> src/docs/src/documentation/content/xdocs/start.xml eedd5b7
> src/org/apache/pig/backend/hadoop/executionengine/spark/SparkLauncher.java b542013
>
> Diff: https://reviews.apache.org/r/39641/diff/
>
>
> Testing
> -------
>
> Verified that the dynamic configuration is hornoured by the yarn system. Requires the auxillary shuffle service need to be enabled at the node manager and application level for this to work correctly.
>
>
> Thanks,
>
> Srikanth Sundarrajan
>
>