You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Swapan Shridhar <ss...@hortonworks.com> on 2017/01/26 01:36:09 UTC
Review Request 55974: AMBARI-19721. Use 'llapstatus' command watch
mode from HDP 2.6 onwards.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55974/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
Bugs: AMBARI-19721
https://issues.apache.org/jira/browse/AMBARI-19721
Repository: ambari
Description
-------
- Till HDP 2.5, Ambari had used *'llapstatus'* command to query LLAP app status and has wrapped around the retry logic for how many times to query *'llapstatus'* command in order for LLAP app to reach *RUNNING_ALL* or *RUNNING* with 80 % containers Up.
- With HDP 2.6, Hive has implemented a watch mode where Ambari doesnt need to have the retry logic in Ambari itself, and can use the *llapstatus* watch mode.
**Usage is as follows:**
Following options are added to llap status tool
**->** -w : Watch mode waits until all LLAP daemons are running or subset of the nodes are running (threshold can be specified via -r option) (Default wait until all nodes are running)
**->** -r : When watch mode is enabled (-w), wait until the specified threshold of nodes are running (Default 1.0 which means 100% nodes are running)
**->** -i : Amount of time in seconds to wait until subsequent status checks in watch mode (Default: 1sec)
**->** -t : Exit watch mode if the desired state is not attained until the specified timeout (Default: 300sec)
**Example usage:**
*
$llapstatus -w -i 1 -t 300
The above command waits until all nodes are running with 1 second refresh rate and 300 second timeout (300 max attempts else fail).
$llapstatus -w -r 0.8 -i 2 -t 150
The above command waits until 80% of the nodes are running with 2 second refresh rate and 150 second timeout (75 max attempts else fail).
$llapstatus -w -r 0.8
Use without providing the -i and -t options (Defaults used).
Diffs
-----
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py 053c009
ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml 0c9ce62
ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py ab6528b
Diff: https://reviews.apache.org/r/55974/diff/
Testing
-------
Yes.
- Python UT added and Tested
- Tested on cluster for HDP 2.5 and 2.6
Thanks,
Swapan Shridhar
Re: Review Request 55974: AMBARI-19721. Use 'llapstatus' command
watch mode from HDP 2.6 onwards.
Posted by Swapan Shridhar <ss...@hortonworks.com>.
> On Jan. 26, 2017, 1:43 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py, line 263
> > <https://reviews.apache.org/r/55974/diff/1/?file=1616066#file1616066line263>
> >
> > Instead of hdp_25 and hdp_26 use llap_tp and llap_ga
Done.
> On Jan. 26, 2017, 1:43 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py, line 429
> > <https://reviews.apache.org/r/55974/diff/1/?file=1616066#file1616066line429>
> >
> > Similar as above - do not use HDP and version in the common code base
Done.
> On Jan. 26, 2017, 1:43 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml, line 63
> > <https://reviews.apache.org/r/55974/diff/1/?file=1616067#file1616067line63>
> >
> > Why should it be auto added as part of Ambari upgrade?
Done. Copy habits die hard :-(
> On Jan. 26, 2017, 1:43 a.m., Sumit Mohanty wrote:
> > ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py, line 817
> > <https://reviews.apache.org/r/55974/diff/1/?file=1616068#file1616068line817>
> >
> > You can probably do a search and replace all occurrences of hdp and version
Done.
- Swapan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55974/#review163072
-----------------------------------------------------------
On Jan. 26, 2017, 1:36 a.m., Swapan Shridhar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55974/
> -----------------------------------------------------------
>
> (Updated Jan. 26, 2017, 1:36 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
>
>
> Bugs: AMBARI-19721
> https://issues.apache.org/jira/browse/AMBARI-19721
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Till HDP 2.5, Ambari had used *'llapstatus'* command to query LLAP app status and has wrapped around the retry logic for how many times to query *'llapstatus'* command in order for LLAP app to reach *RUNNING_ALL* or *RUNNING* with 80 % containers Up.
>
> - With HDP 2.6, Hive has implemented a watch mode where Ambari doesnt need to have the retry logic in Ambari itself, and can use the *llapstatus* watch mode.
>
> **Usage is as follows:**
>
> Following options are added to llap status tool
>
> **->** -w : Watch mode waits until all LLAP daemons are running or subset of the nodes are running (threshold can be specified via -r option) (Default wait until all nodes are running)
> **->** -r : When watch mode is enabled (-w), wait until the specified threshold of nodes are running (Default 1.0 which means 100% nodes are running)
> **->** -i : Amount of time in seconds to wait until subsequent status checks in watch mode (Default: 1sec)
> **->** -t : Exit watch mode if the desired state is not attained until the specified timeout (Default: 300sec)
>
> **Example usage:**
> *
> $llapstatus -w -i 1 -t 300
> The above command waits until all nodes are running with 1 second refresh rate and 300 second timeout (300 max attempts else fail).
>
>
> $llapstatus -w -r 0.8 -i 2 -t 150
> The above command waits until 80% of the nodes are running with 2 second refresh rate and 150 second timeout (75 max attempts else fail).
>
> $llapstatus -w -r 0.8
> Use without providing the -i and -t options (Defaults used).
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py 053c009
> ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml 0c9ce62
> ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py ab6528b
>
> Diff: https://reviews.apache.org/r/55974/diff/
>
>
> Testing
> -------
>
> Yes.
> - Python UT added and Tested
> - Tested on cluster for HDP 2.5 and 2.6
>
>
> Thanks,
>
> Swapan Shridhar
>
>
Re: Review Request 55974: AMBARI-19721. Use 'llapstatus' command
watch mode from HDP 2.6 onwards.
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55974/#review163072
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py (line 263)
<https://reviews.apache.org/r/55974/#comment234505>
Instead of hdp_25 and hdp_26 use llap_tp and llap_ga
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py (line 429)
<https://reviews.apache.org/r/55974/#comment234506>
Similar as above - do not use HDP and version in the common code base
ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml (line 63)
<https://reviews.apache.org/r/55974/#comment234507>
Why should it be auto added as part of Ambari upgrade?
ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py (line 817)
<https://reviews.apache.org/r/55974/#comment234508>
You can probably do a search and replace all occurrences of hdp and version
- Sumit Mohanty
On Jan. 26, 2017, 1:36 a.m., Swapan Shridhar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55974/
> -----------------------------------------------------------
>
> (Updated Jan. 26, 2017, 1:36 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
>
>
> Bugs: AMBARI-19721
> https://issues.apache.org/jira/browse/AMBARI-19721
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Till HDP 2.5, Ambari had used *'llapstatus'* command to query LLAP app status and has wrapped around the retry logic for how many times to query *'llapstatus'* command in order for LLAP app to reach *RUNNING_ALL* or *RUNNING* with 80 % containers Up.
>
> - With HDP 2.6, Hive has implemented a watch mode where Ambari doesnt need to have the retry logic in Ambari itself, and can use the *llapstatus* watch mode.
>
> **Usage is as follows:**
>
> Following options are added to llap status tool
>
> **->** -w : Watch mode waits until all LLAP daemons are running or subset of the nodes are running (threshold can be specified via -r option) (Default wait until all nodes are running)
> **->** -r : When watch mode is enabled (-w), wait until the specified threshold of nodes are running (Default 1.0 which means 100% nodes are running)
> **->** -i : Amount of time in seconds to wait until subsequent status checks in watch mode (Default: 1sec)
> **->** -t : Exit watch mode if the desired state is not attained until the specified timeout (Default: 300sec)
>
> **Example usage:**
> *
> $llapstatus -w -i 1 -t 300
> The above command waits until all nodes are running with 1 second refresh rate and 300 second timeout (300 max attempts else fail).
>
>
> $llapstatus -w -r 0.8 -i 2 -t 150
> The above command waits until 80% of the nodes are running with 2 second refresh rate and 150 second timeout (75 max attempts else fail).
>
> $llapstatus -w -r 0.8
> Use without providing the -i and -t options (Defaults used).
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py 053c009
> ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml 0c9ce62
> ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py ab6528b
>
> Diff: https://reviews.apache.org/r/55974/diff/
>
>
> Testing
> -------
>
> Yes.
> - Python UT added and Tested
> - Tested on cluster for HDP 2.5 and 2.6
>
>
> Thanks,
>
> Swapan Shridhar
>
>
Re: Review Request 55974: AMBARI-19721. Use 'llapstatus' command
watch mode from HDP 2.6 onwards.
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55974/#review163084
-----------------------------------------------------------
Ship it!
Ship It!
- Sumit Mohanty
On Jan. 26, 2017, 2:07 a.m., Swapan Shridhar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55974/
> -----------------------------------------------------------
>
> (Updated Jan. 26, 2017, 2:07 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
>
>
> Bugs: AMBARI-19721
> https://issues.apache.org/jira/browse/AMBARI-19721
>
>
> Repository: ambari
>
>
> Description
> -------
>
> - Till HDP 2.5, Ambari had used *'llapstatus'* command to query LLAP app status and has wrapped around the retry logic for how many times to query *'llapstatus'* command in order for LLAP app to reach *RUNNING_ALL* or *RUNNING* with 80 % containers Up.
>
> - With HDP 2.6, Hive has implemented a watch mode where Ambari doesnt need to have the retry logic in Ambari itself, and can use the *llapstatus* watch mode.
>
> **Usage is as follows:**
>
> Following options are added to llap status tool
>
> **->** -w : Watch mode waits until all LLAP daemons are running or subset of the nodes are running (threshold can be specified via -r option) (Default wait until all nodes are running)
> **->** -r : When watch mode is enabled (-w), wait until the specified threshold of nodes are running (Default 1.0 which means 100% nodes are running)
> **->** -i : Amount of time in seconds to wait until subsequent status checks in watch mode (Default: 1sec)
> **->** -t : Exit watch mode if the desired state is not attained until the specified timeout (Default: 300sec)
>
> **Example usage:**
> *
> $llapstatus -w -i 1 -t 300
> The above command waits until all nodes are running with 1 second refresh rate and 300 second timeout (300 max attempts else fail).
>
>
> $llapstatus -w -r 0.8 -i 2 -t 150
> The above command waits until 80% of the nodes are running with 2 second refresh rate and 150 second timeout (75 max attempts else fail).
>
> $llapstatus -w -r 0.8
> Use without providing the -i and -t options (Defaults used).
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py 053c009
> ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml 0c9ce62
> ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py ab6528b
>
> Diff: https://reviews.apache.org/r/55974/diff/
>
>
> Testing
> -------
>
> Yes.
> - Python UT added and Tested
> - Tested on cluster for HDP 2.5 and 2.6
>
>
> Thanks,
>
> Swapan Shridhar
>
>
Re: Review Request 55974: AMBARI-19721. Use 'llapstatus' command
watch mode from HDP 2.6 onwards.
Posted by Swapan Shridhar <ss...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55974/
-----------------------------------------------------------
(Updated Jan. 26, 2017, 2:07 a.m.)
Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
Changes
-------
Suggested as per review by Sumit.
Bugs: AMBARI-19721
https://issues.apache.org/jira/browse/AMBARI-19721
Repository: ambari
Description
-------
- Till HDP 2.5, Ambari had used *'llapstatus'* command to query LLAP app status and has wrapped around the retry logic for how many times to query *'llapstatus'* command in order for LLAP app to reach *RUNNING_ALL* or *RUNNING* with 80 % containers Up.
- With HDP 2.6, Hive has implemented a watch mode where Ambari doesnt need to have the retry logic in Ambari itself, and can use the *llapstatus* watch mode.
**Usage is as follows:**
Following options are added to llap status tool
**->** -w : Watch mode waits until all LLAP daemons are running or subset of the nodes are running (threshold can be specified via -r option) (Default wait until all nodes are running)
**->** -r : When watch mode is enabled (-w), wait until the specified threshold of nodes are running (Default 1.0 which means 100% nodes are running)
**->** -i : Amount of time in seconds to wait until subsequent status checks in watch mode (Default: 1sec)
**->** -t : Exit watch mode if the desired state is not attained until the specified timeout (Default: 300sec)
**Example usage:**
*
$llapstatus -w -i 1 -t 300
The above command waits until all nodes are running with 1 second refresh rate and 300 second timeout (300 max attempts else fail).
$llapstatus -w -r 0.8 -i 2 -t 150
The above command waits until 80% of the nodes are running with 2 second refresh rate and 150 second timeout (75 max attempts else fail).
$llapstatus -w -r 0.8
Use without providing the -i and -t options (Defaults used).
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py 053c009
ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-env.xml 0c9ce62
ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py ab6528b
Diff: https://reviews.apache.org/r/55974/diff/
Testing
-------
Yes.
- Python UT added and Tested
- Tested on cluster for HDP 2.5 and 2.6
Thanks,
Swapan Shridhar