You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Navina Ramesh <nr...@linkedin.com> on 2015/10/08 09:27:14 UTC
Review Request 39119: SAMZA-792: SamzaAppMaster Java code needs to
pass the requested container memory size to RM
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39119/
-----------------------------------------------------------
Review request for samza, Yan Fang, Chinmay Soman, Jake Maes, and Yi Pan (Data Infrastructure).
Bugs: SAMZA-792
https://issues.apache.org/jira/browse/SAMZA-792
Repository: samza
Description
-------
SAMZA-792: SamzaAppMaster Java code needs to pass the requested container memory size to RM
Diffs
-----
samza-yarn/src/main/java/org/apache/samza/job/yarn/AbstractContainerAllocator.java eec1708571cb361d9c228efa19a14b24a3ae4a8e
samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerAllocator.java 9911540ad65cc75fc7f74f97264573ef2a80dad2
samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java e3b58685084a8643f0ef554a00daadff409a8ffa
samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaContainerRequest.java 9441d772af6971ad7ef4665430cf8fe20ce4f24b
samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaTaskManager.java 12f2f2cba980c82f07b3919771f841dfca7a7945
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestContainerAllocator.java 01f32a47726ef5b8e8512826a3336ddfa7709eaf
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestHostAwareContainerAllocator.java 663ea250e88949da13ce2af7dbecd4cb737e75d5
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java 4c1eaa9354e3e3cfed9bf5e032d6d9e89a9bd8b5
samza-yarn/src/test/java/org/apache/samza/job/yarn/util/MockContainerAllocator.java 85f871a85b8fced212c7418d4c9a7f0de702811e
Diff: https://reviews.apache.org/r/39119/diff/
Testing
-------
Thanks,
Navina Ramesh
Re: Review Request 39119: SAMZA-792: SamzaAppMaster Java code needs
to pass the requested container memory size to RM
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39119/#review101974
-----------------------------------------------------------
Ship it!
Ship It!
- Yi Pan (Data Infrastructure)
On Oct. 8, 2015, 10:07 p.m., Navina Ramesh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39119/
> -----------------------------------------------------------
>
> (Updated Oct. 8, 2015, 10:07 p.m.)
>
>
> Review request for samza, Yan Fang, Chinmay Soman, Jake Maes, and Yi Pan (Data Infrastructure).
>
>
> Bugs: SAMZA-792
> https://issues.apache.org/jira/browse/SAMZA-792
>
>
> Repository: samza
>
>
> Description
> -------
>
> SAMZA-792: SamzaAppMaster Java code needs to pass the requested container memory size to RM
>
>
> Diffs
> -----
>
> samza-yarn/src/main/java/org/apache/samza/job/yarn/AbstractContainerAllocator.java eec1708571cb361d9c228efa19a14b24a3ae4a8e
> samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerAllocator.java 9911540ad65cc75fc7f74f97264573ef2a80dad2
> samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java e3b58685084a8643f0ef554a00daadff409a8ffa
> samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaContainerRequest.java 9441d772af6971ad7ef4665430cf8fe20ce4f24b
> samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaTaskManager.java 12f2f2cba980c82f07b3919771f841dfca7a7945
> samza-yarn/src/test/java/org/apache/samza/job/yarn/TestContainerAllocator.java 01f32a47726ef5b8e8512826a3336ddfa7709eaf
> samza-yarn/src/test/java/org/apache/samza/job/yarn/TestHostAwareContainerAllocator.java 663ea250e88949da13ce2af7dbecd4cb737e75d5
> samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java 4c1eaa9354e3e3cfed9bf5e032d6d9e89a9bd8b5
> samza-yarn/src/test/java/org/apache/samza/job/yarn/util/MockContainerAllocator.java 85f871a85b8fced212c7418d4c9a7f0de702811e
>
> Diff: https://reviews.apache.org/r/39119/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Navina Ramesh
>
>
Re: Review Request 39119: SAMZA-792: SamzaAppMaster Java code needs
to pass the requested container memory size to RM
Posted by Navina Ramesh <nr...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39119/
-----------------------------------------------------------
(Updated Oct. 8, 2015, 10:07 p.m.)
Review request for samza, Yan Fang, Chinmay Soman, Jake Maes, and Yi Pan (Data Infrastructure).
Changes
-------
Added a simple test in unit test
Bugs: SAMZA-792
https://issues.apache.org/jira/browse/SAMZA-792
Repository: samza
Description
-------
SAMZA-792: SamzaAppMaster Java code needs to pass the requested container memory size to RM
Diffs (updated)
-----
samza-yarn/src/main/java/org/apache/samza/job/yarn/AbstractContainerAllocator.java eec1708571cb361d9c228efa19a14b24a3ae4a8e
samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerAllocator.java 9911540ad65cc75fc7f74f97264573ef2a80dad2
samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java e3b58685084a8643f0ef554a00daadff409a8ffa
samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaContainerRequest.java 9441d772af6971ad7ef4665430cf8fe20ce4f24b
samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaTaskManager.java 12f2f2cba980c82f07b3919771f841dfca7a7945
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestContainerAllocator.java 01f32a47726ef5b8e8512826a3336ddfa7709eaf
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestHostAwareContainerAllocator.java 663ea250e88949da13ce2af7dbecd4cb737e75d5
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java 4c1eaa9354e3e3cfed9bf5e032d6d9e89a9bd8b5
samza-yarn/src/test/java/org/apache/samza/job/yarn/util/MockContainerAllocator.java 85f871a85b8fced212c7418d4c9a7f0de702811e
Diff: https://reviews.apache.org/r/39119/diff/
Testing
-------
Thanks,
Navina Ramesh
Re: Review Request 39119: SAMZA-792: SamzaAppMaster Java code needs
to pass the requested container memory size to RM
Posted by "Yi Pan (Data Infrastructure)" <yi...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39119/#review101950
-----------------------------------------------------------
Ship it!
LGTM. Just a minor comment in test cases.
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java (line 172)
<https://reviews.apache.org/r/39119/#comment159461>
It would be nice to add a unit test here to verify that the SamzaContainerRequest actually uses the configured request parameters in YarnConfig, instead of the default ones.
- Yi Pan (Data Infrastructure)
On Oct. 8, 2015, 7:39 a.m., Navina Ramesh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39119/
> -----------------------------------------------------------
>
> (Updated Oct. 8, 2015, 7:39 a.m.)
>
>
> Review request for samza, Yan Fang, Chinmay Soman, Jake Maes, and Yi Pan (Data Infrastructure).
>
>
> Bugs: SAMZA-792
> https://issues.apache.org/jira/browse/SAMZA-792
>
>
> Repository: samza
>
>
> Description
> -------
>
> SAMZA-792: SamzaAppMaster Java code needs to pass the requested container memory size to RM
>
>
> Diffs
> -----
>
> samza-yarn/src/main/java/org/apache/samza/job/yarn/AbstractContainerAllocator.java eec1708571cb361d9c228efa19a14b24a3ae4a8e
> samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerAllocator.java 9911540ad65cc75fc7f74f97264573ef2a80dad2
> samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java e3b58685084a8643f0ef554a00daadff409a8ffa
> samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaContainerRequest.java 9441d772af6971ad7ef4665430cf8fe20ce4f24b
> samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaTaskManager.java 12f2f2cba980c82f07b3919771f841dfca7a7945
> samza-yarn/src/test/java/org/apache/samza/job/yarn/TestContainerAllocator.java 01f32a47726ef5b8e8512826a3336ddfa7709eaf
> samza-yarn/src/test/java/org/apache/samza/job/yarn/TestHostAwareContainerAllocator.java 663ea250e88949da13ce2af7dbecd4cb737e75d5
> samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java 4c1eaa9354e3e3cfed9bf5e032d6d9e89a9bd8b5
> samza-yarn/src/test/java/org/apache/samza/job/yarn/util/MockContainerAllocator.java 85f871a85b8fced212c7418d4c9a7f0de702811e
>
> Diff: https://reviews.apache.org/r/39119/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Navina Ramesh
>
>
Re: Review Request 39119: SAMZA-792: SamzaAppMaster Java code needs
to pass the requested container memory size to RM
Posted by Navina Ramesh <nr...@linkedin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39119/
-----------------------------------------------------------
(Updated Oct. 8, 2015, 7:39 a.m.)
Review request for samza, Yan Fang, Chinmay Soman, Jake Maes, and Yi Pan (Data Infrastructure).
Bugs: SAMZA-792
https://issues.apache.org/jira/browse/SAMZA-792
Repository: samza
Description
-------
SAMZA-792: SamzaAppMaster Java code needs to pass the requested container memory size to RM
Diffs (updated)
-----
samza-yarn/src/main/java/org/apache/samza/job/yarn/AbstractContainerAllocator.java eec1708571cb361d9c228efa19a14b24a3ae4a8e
samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerAllocator.java 9911540ad65cc75fc7f74f97264573ef2a80dad2
samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java e3b58685084a8643f0ef554a00daadff409a8ffa
samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaContainerRequest.java 9441d772af6971ad7ef4665430cf8fe20ce4f24b
samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaTaskManager.java 12f2f2cba980c82f07b3919771f841dfca7a7945
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestContainerAllocator.java 01f32a47726ef5b8e8512826a3336ddfa7709eaf
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestHostAwareContainerAllocator.java 663ea250e88949da13ce2af7dbecd4cb737e75d5
samza-yarn/src/test/java/org/apache/samza/job/yarn/TestSamzaTaskManager.java 4c1eaa9354e3e3cfed9bf5e032d6d9e89a9bd8b5
samza-yarn/src/test/java/org/apache/samza/job/yarn/util/MockContainerAllocator.java 85f871a85b8fced212c7418d4c9a7f0de702811e
Diff: https://reviews.apache.org/r/39119/diff/
Testing
-------
Thanks,
Navina Ramesh