You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Sid Wagle <sw...@hortonworks.com> on 2014/02/20 00:09:23 UTC

Review Request 18292: Upload tez libraries to HDFS on Tez install

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/
-----------------------------------------------------------

Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Mahadev Konar, and Sumit Mohanty.


Bugs: AMBARI-4754
    https://issues.apache.org/jira/browse/AMBARI-4754


Repository: ambari


Description
-------

1. Change default location of tez libraries to HDFS apps directory
2. Create directory on Tez install and upload tez jars.
3. Modify tez-site.xml


Diffs
-----

  ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
  ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
  ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 

Diff: https://reviews.apache.org/r/18292/diff/


Testing
-------


Thanks,

Sid Wagle

Re: Review Request 18292: Upload tez libraries to HDFS on Tez install

Posted by Sid Wagle <sw...@hortonworks.com>.

> On Feb. 19, 2014, 11:41 p.m., Sumit Mohanty wrote:
> > Can copyFromLocal be part of the resource management library?

Good point. Currently it seems the python library does not deal with any explicit hadoop commands. Maybe Andrew / Dmitry L can shed some light.


- Sid


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/#review34945
-----------------------------------------------------------


On Feb. 20, 2014, 12:02 a.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18292/
> -----------------------------------------------------------
> 
> (Updated Feb. 20, 2014, 12:02 a.m.)
> 
> 
> Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Dmitro Lisnichenko, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-4754
>     https://issues.apache.org/jira/browse/AMBARI-4754
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1. Change default location of tez libraries to HDFS apps directory
> 2. Create directory on Tez install and upload tez jars.
> 3. Modify tez-site.xml
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
>   ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
>   ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 
> 
> Diff: https://reviews.apache.org/r/18292/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 18292: Upload tez libraries to HDFS on Tez install

Posted by Sumit Mohanty <sm...@hortonworks.com>.

> On Feb. 19, 2014, 11:41 p.m., Sumit Mohanty wrote:
> > Can copyFromLocal be part of the resource management library?
> 
> Sid Wagle wrote:
>     Good point. Currently it seems the python library does not deal with any explicit hadoop commands. Maybe Andrew / Dmitry L can shed some light.
> 
> Sid Wagle wrote:
>     Created a Jira to address this separately.
>     
>     https://issues.apache.org/jira/browse/AMBARI-4767

look at .../ambari-agent/src/main/python/resource_management/libraries/providers/execute_hadoop.py. We may be able to add it there.


- Sumit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/#review34945
-----------------------------------------------------------


On Feb. 20, 2014, 12:02 a.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18292/
> -----------------------------------------------------------
> 
> (Updated Feb. 20, 2014, 12:02 a.m.)
> 
> 
> Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Dmitro Lisnichenko, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-4754
>     https://issues.apache.org/jira/browse/AMBARI-4754
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1. Change default location of tez libraries to HDFS apps directory
> 2. Create directory on Tez install and upload tez jars.
> 3. Modify tez-site.xml
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
>   ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
>   ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 
> 
> Diff: https://reviews.apache.org/r/18292/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 18292: Upload tez libraries to HDFS on Tez install

Posted by Sid Wagle <sw...@hortonworks.com>.

> On Feb. 19, 2014, 11:41 p.m., Sumit Mohanty wrote:
> > Can copyFromLocal be part of the resource management library?
> 
> Sid Wagle wrote:
>     Good point. Currently it seems the python library does not deal with any explicit hadoop commands. Maybe Andrew / Dmitry L can shed some light.

Created a Jira to address this separately.

https://issues.apache.org/jira/browse/AMBARI-4767


- Sid


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/#review34945
-----------------------------------------------------------


On Feb. 20, 2014, 12:02 a.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18292/
> -----------------------------------------------------------
> 
> (Updated Feb. 20, 2014, 12:02 a.m.)
> 
> 
> Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Dmitro Lisnichenko, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-4754
>     https://issues.apache.org/jira/browse/AMBARI-4754
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1. Change default location of tez libraries to HDFS apps directory
> 2. Create directory on Tez install and upload tez jars.
> 3. Modify tez-site.xml
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
>   ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
>   ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 
> 
> Diff: https://reviews.apache.org/r/18292/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 18292: Upload tez libraries to HDFS on Tez install

Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/#review34945
-----------------------------------------------------------


Can copyFromLocal be part of the resource management library?

- Sumit Mohanty


On Feb. 19, 2014, 11:09 p.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18292/
> -----------------------------------------------------------
> 
> (Updated Feb. 19, 2014, 11:09 p.m.)
> 
> 
> Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-4754
>     https://issues.apache.org/jira/browse/AMBARI-4754
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1. Change default location of tez libraries to HDFS apps directory
> 2. Create directory on Tez install and upload tez jars.
> 3. Modify tez-site.xml
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
>   ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
>   ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 
> 
> Diff: https://reviews.apache.org/r/18292/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 18292: Upload tez libraries to HDFS on Tez install

Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/#review35039
-----------------------------------------------------------

Ship it!


Ship It!

- Sumit Mohanty


On Feb. 20, 2014, 12:02 a.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18292/
> -----------------------------------------------------------
> 
> (Updated Feb. 20, 2014, 12:02 a.m.)
> 
> 
> Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Dmitro Lisnichenko, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-4754
>     https://issues.apache.org/jira/browse/AMBARI-4754
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1. Change default location of tez libraries to HDFS apps directory
> 2. Create directory on Tez install and upload tez jars.
> 3. Modify tez-site.xml
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
>   ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
>   ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 
> 
> Diff: https://reviews.apache.org/r/18292/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 18292: Upload tez libraries to HDFS on Tez install

Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/#review35262
-----------------------------------------------------------



ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py
<https://reviews.apache.org/r/18292/#comment65724>

    You forgot to uncomment it back, this breaks early failing, that's just for the next time not to forget ;)


- Andrew Onischuk


On Feb. 20, 2014, 12:02 a.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18292/
> -----------------------------------------------------------
> 
> (Updated Feb. 20, 2014, 12:02 a.m.)
> 
> 
> Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Dmitro Lisnichenko, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-4754
>     https://issues.apache.org/jira/browse/AMBARI-4754
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1. Change default location of tez libraries to HDFS apps directory
> 2. Create directory on Tez install and upload tez jars.
> 3. Modify tez-site.xml
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
>   ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
>   ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 
> 
> Diff: https://reviews.apache.org/r/18292/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 18292: Upload tez libraries to HDFS on Tez install

Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/#review35035
-----------------------------------------------------------

Ship it!


Ship It!

- Dmitro Lisnichenko


On Feb. 20, 2014, 12:02 a.m., Sid Wagle wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18292/
> -----------------------------------------------------------
> 
> (Updated Feb. 20, 2014, 12:02 a.m.)
> 
> 
> Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Dmitro Lisnichenko, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-4754
>     https://issues.apache.org/jira/browse/AMBARI-4754
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> 1. Change default location of tez libraries to HDFS apps directory
> 2. Create directory on Tez install and upload tez jars.
> 3. Modify tez-site.xml
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
>   ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
>   ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
>   ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 
> 
> Diff: https://reviews.apache.org/r/18292/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 18292: Upload tez libraries to HDFS on Tez install

Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18292/
-----------------------------------------------------------

(Updated Feb. 20, 2014, 12:02 a.m.)


Review request for Ambari, Andrew-Duplicate-Accct-Inactiv Konwinski, Dmitro Lisnichenko, Mahadev Konar, and Sumit Mohanty.


Bugs: AMBARI-4754
    https://issues.apache.org/jira/browse/AMBARI-4754


Repository: ambari


Description
-------

1. Change default location of tez libraries to HDFS apps directory
2. Create directory on Tez install and upload tez jars.
3. Modify tez-site.xml


Diffs
-----

  ambari-agent/src/main/python/resource_management/libraries/script/config_dictionary.py 453c546 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/configuration/tez-site.xml 5d78c08 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/params.py 203fc66 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/scripts/tez.py b23a356 
  ambari-server/src/main/resources/stacks/HDP/2.1.1/services/TEZ/package/templates/tez-env.sh.j2 6fbadff 
  ambari-server/src/test/python/stacks/2.1.1/TEZ/test_tez_client.py 05224fc 
  ambari-server/src/test/python/stacks/2.1.1/configs/default.json ebb0742 

Diff: https://reviews.apache.org/r/18292/diff/


Testing
-------


Thanks,

Sid Wagle