You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2014/04/16 19:52:30 UTC

Review Request 20420: Ambari should Manage JobHistory Heap Size

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20420/
-----------------------------------------------------------

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-5485
    https://issues.apache.org/jira/browse/AMBARI-5485


Repository: ambari


Description
-------

PROBLEM: Ambari does not manage mapred-env.sh and JobHistory memory cannot be
set via Ambari.

BUSINESS IMPACT: Users are not sure how to change JobHistory memory settings
in Ambari cluster. This came from eBay so looks like large clusters need to be
able to increase memory.

STEPS TO REPRODUCE: navigate to Services -> Mapreduce -> Configs tab and you
will notice that JobHistory Heap configuration is not there.

ACTUAL BEHAVIOR: Cannot change JobHistory memory in Ambari UI

EXPECTED BEHAVIOR: To be able to manage JobHistory memory in Ambari UI

SUPPORT ANALYSIS: Since Ambari does not manage mapred-env.sh you can manually
edit config in mapred-env.sh. I verified by manually setting the memory and
restarting cluster via Ambari that config value did not change.

    
    
    
    [root@63ambarihdp2010 conf]# grep -i heap /etc/hadoop/conf/mapred-env.sh
    export HADOOP_JOB_HISTORYSERVER_HEAPSIZE=900
    


Diffs
-----

  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py f1ef84a 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 3ffcc76 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/mapred-env.sh.j2 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration-mapred/global.xml 984cd41 
  ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 88448e6 
  ambari-web/app/controllers/main/service/info/configs.js c183b83 
  ambari-web/app/controllers/wizard/step8_controller.js 8bb548b 
  ambari-web/app/data/HDP2/global_properties.js 0b61e77 

Diff: https://reviews.apache.org/r/20420/diff/


Testing
-------


Thanks,

Andrew Onischuk

Re: Review Request 20420: Ambari should Manage JobHistory Heap Size

Posted by Andrew Onischuk <ao...@hortonworks.com>.

> On April 16, 2014, 6:06 p.m., Andrew Onischuk wrote:
> > have also to add the same changes for HDP1.

nevermind


- Andrew


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20420/#review40580
-----------------------------------------------------------


On April 16, 2014, 5:52 p.m., Andrew Onischuk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20420/
> -----------------------------------------------------------
> 
> (Updated April 16, 2014, 5:52 p.m.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-5485
>     https://issues.apache.org/jira/browse/AMBARI-5485
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: Ambari does not manage mapred-env.sh and JobHistory memory cannot be
> set via Ambari.
> 
> BUSINESS IMPACT: Users are not sure how to change JobHistory memory settings
> in Ambari cluster. This came from eBay so looks like large clusters need to be
> able to increase memory.
> 
> STEPS TO REPRODUCE: navigate to Services -> Mapreduce -> Configs tab and you
> will notice that JobHistory Heap configuration is not there.
> 
> ACTUAL BEHAVIOR: Cannot change JobHistory memory in Ambari UI
> 
> EXPECTED BEHAVIOR: To be able to manage JobHistory memory in Ambari UI
> 
> SUPPORT ANALYSIS: Since Ambari does not manage mapred-env.sh you can manually
> edit config in mapred-env.sh. I verified by manually setting the memory and
> restarting cluster via Ambari that config value did not change.
> 
>     
>     
>     
>     [root@63ambarihdp2010 conf]# grep -i heap /etc/hadoop/conf/mapred-env.sh
>     export HADOOP_JOB_HISTORYSERVER_HEAPSIZE=900
>     
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py f1ef84a 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 3ffcc76 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/mapred-env.sh.j2 PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration-mapred/global.xml 984cd41 
>   ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 88448e6 
>   ambari-web/app/controllers/main/service/info/configs.js c183b83 
>   ambari-web/app/controllers/wizard/step8_controller.js 8bb548b 
>   ambari-web/app/data/HDP2/global_properties.js 0b61e77 
> 
> Diff: https://reviews.apache.org/r/20420/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 20420: Ambari should Manage JobHistory Heap Size

Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20420/#review40580
-----------------------------------------------------------


have also to add the same changes for HDP1.

- Andrew Onischuk


On April 16, 2014, 5:52 p.m., Andrew Onischuk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20420/
> -----------------------------------------------------------
> 
> (Updated April 16, 2014, 5:52 p.m.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-5485
>     https://issues.apache.org/jira/browse/AMBARI-5485
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: Ambari does not manage mapred-env.sh and JobHistory memory cannot be
> set via Ambari.
> 
> BUSINESS IMPACT: Users are not sure how to change JobHistory memory settings
> in Ambari cluster. This came from eBay so looks like large clusters need to be
> able to increase memory.
> 
> STEPS TO REPRODUCE: navigate to Services -> Mapreduce -> Configs tab and you
> will notice that JobHistory Heap configuration is not there.
> 
> ACTUAL BEHAVIOR: Cannot change JobHistory memory in Ambari UI
> 
> EXPECTED BEHAVIOR: To be able to manage JobHistory memory in Ambari UI
> 
> SUPPORT ANALYSIS: Since Ambari does not manage mapred-env.sh you can manually
> edit config in mapred-env.sh. I verified by manually setting the memory and
> restarting cluster via Ambari that config value did not change.
> 
>     
>     
>     
>     [root@63ambarihdp2010 conf]# grep -i heap /etc/hadoop/conf/mapred-env.sh
>     export HADOOP_JOB_HISTORYSERVER_HEAPSIZE=900
>     
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py f1ef84a 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 3ffcc76 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/mapred-env.sh.j2 PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration-mapred/global.xml 984cd41 
>   ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 88448e6 
>   ambari-web/app/controllers/main/service/info/configs.js c183b83 
>   ambari-web/app/controllers/wizard/step8_controller.js 8bb548b 
>   ambari-web/app/data/HDP2/global_properties.js 0b61e77 
> 
> Diff: https://reviews.apache.org/r/20420/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 20420: Ambari should Manage JobHistory Heap Size

Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20420/#review40648
-----------------------------------------------------------

Ship it!


Ship It!

- Dmytro Sen


On April 17, 2014, 2:42 p.m., Andrew Onischuk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20420/
> -----------------------------------------------------------
> 
> (Updated April 17, 2014, 2:42 p.m.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-5485
>     https://issues.apache.org/jira/browse/AMBARI-5485
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> PROBLEM: Ambari does not manage mapred-env.sh and JobHistory memory cannot be
> set via Ambari.
> 
> BUSINESS IMPACT: Users are not sure how to change JobHistory memory settings
> in Ambari cluster. This came from eBay so looks like large clusters need to be
> able to increase memory.
> 
> STEPS TO REPRODUCE: navigate to Services -> Mapreduce -> Configs tab and you
> will notice that JobHistory Heap configuration is not there.
> 
> ACTUAL BEHAVIOR: Cannot change JobHistory memory in Ambari UI
> 
> EXPECTED BEHAVIOR: To be able to manage JobHistory memory in Ambari UI
> 
> SUPPORT ANALYSIS: Since Ambari does not manage mapred-env.sh you can manually
> edit config in mapred-env.sh. I verified by manually setting the memory and
> restarting cluster via Ambari that config value did not change.
> 
>     
>     
>     
>     [root@63ambarihdp2010 conf]# grep -i heap /etc/hadoop/conf/mapred-env.sh
>     export HADOOP_JOB_HISTORYSERVER_HEAPSIZE=900
>     
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 8b342f7 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 6e34a95 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/mapred-env.sh.j2 PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration-mapred/global.xml 984cd41 
>   ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9841c3f 
>   ambari-web/app/controllers/main/service/info/configs.js c183b83 
>   ambari-web/app/controllers/wizard/step8_controller.js abc617e 
>   ambari-web/app/data/HDP2/global_properties.js 0b61e77 
> 
> Diff: https://reviews.apache.org/r/20420/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 20420: Ambari should Manage JobHistory Heap Size

Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20420/
-----------------------------------------------------------

(Updated April 17, 2014, 2:42 p.m.)


Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-5485
    https://issues.apache.org/jira/browse/AMBARI-5485


Repository: ambari


Description
-------

PROBLEM: Ambari does not manage mapred-env.sh and JobHistory memory cannot be
set via Ambari.

BUSINESS IMPACT: Users are not sure how to change JobHistory memory settings
in Ambari cluster. This came from eBay so looks like large clusters need to be
able to increase memory.

STEPS TO REPRODUCE: navigate to Services -> Mapreduce -> Configs tab and you
will notice that JobHistory Heap configuration is not there.

ACTUAL BEHAVIOR: Cannot change JobHistory memory in Ambari UI

EXPECTED BEHAVIOR: To be able to manage JobHistory memory in Ambari UI

SUPPORT ANALYSIS: Since Ambari does not manage mapred-env.sh you can manually
edit config in mapred-env.sh. I verified by manually setting the memory and
restarting cluster via Ambari that config value did not change.

    
    
    
    [root@63ambarihdp2010 conf]# grep -i heap /etc/hadoop/conf/mapred-env.sh
    export HADOOP_JOB_HISTORYSERVER_HEAPSIZE=900
    


Diffs (updated)
-----

  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py 8b342f7 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 6e34a95 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/mapred-env.sh.j2 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/configuration-mapred/global.xml 984cd41 
  ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 9841c3f 
  ambari-web/app/controllers/main/service/info/configs.js c183b83 
  ambari-web/app/controllers/wizard/step8_controller.js abc617e 
  ambari-web/app/data/HDP2/global_properties.js 0b61e77 

Diff: https://reviews.apache.org/r/20420/diff/


Testing
-------


Thanks,

Andrew Onischuk