You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Emil Anca <ea...@hortonworks.com> on 2015/04/17 17:15:43 UTC
Review Request 33313: Issues when log/pid dirs are customized - Kafka
and Knox
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33313/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk and Robert Levas.
Bugs: AMBARI-10562
https://issues.apache.org/jira/browse/AMBARI-10562
Repository: ambari
Description
-------
Problem:
Customization of the log and pid dir location using ambari, does not work with Kafka and Knox.
After the install wizard error out I also noticed: (all Kafka/Knox pid/log dir paths have a /mydisk/ custom location prefix appended)
* Starting Kafka/Knox from Ambari actually starts the processes but Ambari thinks they died and goes red. On further investigation the /mydisk/run/kafka and /mydisk/run/knox are empty so the pid files are not getting created. I re-installed on fresh VM and ran the below to give kafka/knox users access to those dirs before running install wizard but still got same error (even though as those users I can write to those dirs)
chmod +x /mydisk/run
chmod +x /mydisk
Solution:
Since both Kafka and Knox come with bundled bash scripts that manages their process lifecycle amongst other things and we have no control over them, an agreed solution with was to use symlinks.
For example Kafka uses /usr/hdp/current/kafka-broker/bin/kafka bash to start, where values are hardcoded as such
PID_DIR="/var/run/$APP_NAME" on which we have no control over as they are bundled in the Kafka package.
Knox already uses symlinks by default, all we had to do is change said symlink with the ambari managed location.
Diffs
-----
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py 0170116
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py 419639e
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py 9e20499
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params.py 13b0868
ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 8ffbd67
Diff: https://reviews.apache.org/r/33313/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [6.912s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.116s]
[INFO] Ambari Web ........................................ SUCCESS [29.655s]
[INFO] Ambari Views ...................................... SUCCESS [1.196s]
[INFO] Ambari Admin View ................................. SUCCESS [19.873s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.116s]
[INFO] Ambari Server ..................................... SUCCESS [2:39.489s]
[INFO] Ambari Agent ...................................... SUCCESS [9.179s]
[INFO] Ambari Client ..................................... SUCCESS [0.202s]
[INFO] Ambari Python Client .............................. SUCCESS [1.121s]
[INFO] Ambari Groovy Client .............................. SUCCESS [2.917s]
[INFO] Ambari Shell ...................................... SUCCESS [0.108s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.637s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [1.760s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:55.047s
[INFO] Finished at: Fri Apr 17 15:31:16 EEST 2015
[INFO] Final Memory: 108M/1125M
[INFO] ------------------------------------------------------------------------
Thanks,
Emil Anca
Re: Review Request 33313: Issues when log/pid dirs are customized -
Kafka and Knox
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33313/#review81469
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On April 24, 2015, 12:23 p.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33313/
> -----------------------------------------------------------
>
> (Updated April 24, 2015, 12:23 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk and Robert Levas.
>
>
> Bugs: AMBARI-10562
> https://issues.apache.org/jira/browse/AMBARI-10562
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Problem:
>
> Customization of the log and pid dir location using ambari, does not work with Kafka and Knox.
>
>
> After the install wizard error out I also noticed: (all Kafka/Knox pid/log dir paths have a /mydisk/ custom location prefix appended)
>
> * Starting Kafka/Knox from Ambari actually starts the processes but Ambari thinks they died and goes red. On further investigation the /mydisk/run/kafka and /mydisk/run/knox are empty so the pid files are not getting created. I re-installed on fresh VM and ran the below to give kafka/knox users access to those dirs before running install wizard but still got same error (even though as those users I can write to those dirs)
>
> chmod +x /mydisk/run
> chmod +x /mydisk
>
> Solution:
>
> Since both Kafka and Knox come with bundled bash scripts that manages their process lifecycle amongst other things and we have no control over them, an agreed solution with was to use symlinks.
>
> For example Kafka uses /usr/hdp/current/kafka-broker/bin/kafka bash to start, where values are hardcoded as such
> PID_DIR="/var/run/$APP_NAME" on which we have no control over as they are bundled in the Kafka package.
>
> Knox already uses symlinks by default, all we had to do is change said symlink with the ambari managed location.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py 0170116
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py 74c189e
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py 9e20499
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params.py a04842e
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 2b6f7e4
> ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 37bb66b
> ambari-server/src/test/python/stacks/2.2/configs/default_custom_path_config.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/33313/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [6.912s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.116s]
> [INFO] Ambari Web ........................................ SUCCESS [29.655s]
> [INFO] Ambari Views ...................................... SUCCESS [1.196s]
> [INFO] Ambari Admin View ................................. SUCCESS [19.873s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.116s]
> [INFO] Ambari Server ..................................... SUCCESS [2:39.489s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.179s]
> [INFO] Ambari Client ..................................... SUCCESS [0.202s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.121s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.917s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.108s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.637s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.760s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3:55.047s
> [INFO] Finished at: Fri Apr 17 15:31:16 EEST 2015
> [INFO] Final Memory: 108M/1125M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 33313: Issues when log/pid dirs are customized -
Kafka and Knox
Posted by Emil Anca <ea...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33313/
-----------------------------------------------------------
(Updated April 24, 2015, 12:23 p.m.)
Review request for Ambari, Andrew Onischuk and Robert Levas.
Changes
-------
Reworked patch, went stale.
Bugs: AMBARI-10562
https://issues.apache.org/jira/browse/AMBARI-10562
Repository: ambari
Description
-------
Problem:
Customization of the log and pid dir location using ambari, does not work with Kafka and Knox.
After the install wizard error out I also noticed: (all Kafka/Knox pid/log dir paths have a /mydisk/ custom location prefix appended)
* Starting Kafka/Knox from Ambari actually starts the processes but Ambari thinks they died and goes red. On further investigation the /mydisk/run/kafka and /mydisk/run/knox are empty so the pid files are not getting created. I re-installed on fresh VM and ran the below to give kafka/knox users access to those dirs before running install wizard but still got same error (even though as those users I can write to those dirs)
chmod +x /mydisk/run
chmod +x /mydisk
Solution:
Since both Kafka and Knox come with bundled bash scripts that manages their process lifecycle amongst other things and we have no control over them, an agreed solution with was to use symlinks.
For example Kafka uses /usr/hdp/current/kafka-broker/bin/kafka bash to start, where values are hardcoded as such
PID_DIR="/var/run/$APP_NAME" on which we have no control over as they are bundled in the Kafka package.
Knox already uses symlinks by default, all we had to do is change said symlink with the ambari managed location.
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py 0170116
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py 74c189e
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py 9e20499
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params.py a04842e
ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 2b6f7e4
ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 37bb66b
ambari-server/src/test/python/stacks/2.2/configs/default_custom_path_config.json PRE-CREATION
Diff: https://reviews.apache.org/r/33313/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [6.912s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.116s]
[INFO] Ambari Web ........................................ SUCCESS [29.655s]
[INFO] Ambari Views ...................................... SUCCESS [1.196s]
[INFO] Ambari Admin View ................................. SUCCESS [19.873s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.116s]
[INFO] Ambari Server ..................................... SUCCESS [2:39.489s]
[INFO] Ambari Agent ...................................... SUCCESS [9.179s]
[INFO] Ambari Client ..................................... SUCCESS [0.202s]
[INFO] Ambari Python Client .............................. SUCCESS [1.121s]
[INFO] Ambari Groovy Client .............................. SUCCESS [2.917s]
[INFO] Ambari Shell ...................................... SUCCESS [0.108s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.637s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [1.760s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:55.047s
[INFO] Finished at: Fri Apr 17 15:31:16 EEST 2015
[INFO] Final Memory: 108M/1125M
[INFO] ------------------------------------------------------------------------
Thanks,
Emil Anca
Re: Review Request 33313: Issues when log/pid dirs are customized -
Kafka and Knox
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33313/#review81158
-----------------------------------------------------------
Ship it!
Ship It!
- Robert Levas
On April 22, 2015, 8:47 a.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33313/
> -----------------------------------------------------------
>
> (Updated April 22, 2015, 8:47 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk and Robert Levas.
>
>
> Bugs: AMBARI-10562
> https://issues.apache.org/jira/browse/AMBARI-10562
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Problem:
>
> Customization of the log and pid dir location using ambari, does not work with Kafka and Knox.
>
>
> After the install wizard error out I also noticed: (all Kafka/Knox pid/log dir paths have a /mydisk/ custom location prefix appended)
>
> * Starting Kafka/Knox from Ambari actually starts the processes but Ambari thinks they died and goes red. On further investigation the /mydisk/run/kafka and /mydisk/run/knox are empty so the pid files are not getting created. I re-installed on fresh VM and ran the below to give kafka/knox users access to those dirs before running install wizard but still got same error (even though as those users I can write to those dirs)
>
> chmod +x /mydisk/run
> chmod +x /mydisk
>
> Solution:
>
> Since both Kafka and Knox come with bundled bash scripts that manages their process lifecycle amongst other things and we have no control over them, an agreed solution with was to use symlinks.
>
> For example Kafka uses /usr/hdp/current/kafka-broker/bin/kafka bash to start, where values are hardcoded as such
> PID_DIR="/var/run/$APP_NAME" on which we have no control over as they are bundled in the Kafka package.
>
> Knox already uses symlinks by default, all we had to do is change said symlink with the ambari managed location.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py 0170116
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py 74c189e
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py 9e20499
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params.py a04842e
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py a8b6693
> ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 8ffbd67
> ambari-server/src/test/python/stacks/2.2/configs/default_custom_path_config.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/33313/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [6.912s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.116s]
> [INFO] Ambari Web ........................................ SUCCESS [29.655s]
> [INFO] Ambari Views ...................................... SUCCESS [1.196s]
> [INFO] Ambari Admin View ................................. SUCCESS [19.873s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.116s]
> [INFO] Ambari Server ..................................... SUCCESS [2:39.489s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.179s]
> [INFO] Ambari Client ..................................... SUCCESS [0.202s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.121s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.917s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.108s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.637s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.760s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3:55.047s
> [INFO] Finished at: Fri Apr 17 15:31:16 EEST 2015
> [INFO] Final Memory: 108M/1125M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 33313: Issues when log/pid dirs are customized -
Kafka and Knox
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33313/#review81291
-----------------------------------------------------------
Ship it!
Ship It!
- Andrew Onischuk
On April 22, 2015, 12:47 p.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33313/
> -----------------------------------------------------------
>
> (Updated April 22, 2015, 12:47 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk and Robert Levas.
>
>
> Bugs: AMBARI-10562
> https://issues.apache.org/jira/browse/AMBARI-10562
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Problem:
>
> Customization of the log and pid dir location using ambari, does not work with Kafka and Knox.
>
>
> After the install wizard error out I also noticed: (all Kafka/Knox pid/log dir paths have a /mydisk/ custom location prefix appended)
>
> * Starting Kafka/Knox from Ambari actually starts the processes but Ambari thinks they died and goes red. On further investigation the /mydisk/run/kafka and /mydisk/run/knox are empty so the pid files are not getting created. I re-installed on fresh VM and ran the below to give kafka/knox users access to those dirs before running install wizard but still got same error (even though as those users I can write to those dirs)
>
> chmod +x /mydisk/run
> chmod +x /mydisk
>
> Solution:
>
> Since both Kafka and Knox come with bundled bash scripts that manages their process lifecycle amongst other things and we have no control over them, an agreed solution with was to use symlinks.
>
> For example Kafka uses /usr/hdp/current/kafka-broker/bin/kafka bash to start, where values are hardcoded as such
> PID_DIR="/var/run/$APP_NAME" on which we have no control over as they are bundled in the Kafka package.
>
> Knox already uses symlinks by default, all we had to do is change said symlink with the ambari managed location.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py 0170116
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py 74c189e
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py 9e20499
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params.py a04842e
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py a8b6693
> ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 8ffbd67
> ambari-server/src/test/python/stacks/2.2/configs/default_custom_path_config.json PRE-CREATION
>
> Diff: https://reviews.apache.org/r/33313/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [6.912s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.116s]
> [INFO] Ambari Web ........................................ SUCCESS [29.655s]
> [INFO] Ambari Views ...................................... SUCCESS [1.196s]
> [INFO] Ambari Admin View ................................. SUCCESS [19.873s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.116s]
> [INFO] Ambari Server ..................................... SUCCESS [2:39.489s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.179s]
> [INFO] Ambari Client ..................................... SUCCESS [0.202s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.121s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.917s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.108s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.637s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.760s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3:55.047s
> [INFO] Finished at: Fri Apr 17 15:31:16 EEST 2015
> [INFO] Final Memory: 108M/1125M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 33313: Issues when log/pid dirs are customized -
Kafka and Knox
Posted by Emil Anca <ea...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33313/
-----------------------------------------------------------
(Updated April 22, 2015, 12:47 p.m.)
Review request for Ambari, Andrew Onischuk and Robert Levas.
Bugs: AMBARI-10562
https://issues.apache.org/jira/browse/AMBARI-10562
Repository: ambari
Description
-------
Problem:
Customization of the log and pid dir location using ambari, does not work with Kafka and Knox.
After the install wizard error out I also noticed: (all Kafka/Knox pid/log dir paths have a /mydisk/ custom location prefix appended)
* Starting Kafka/Knox from Ambari actually starts the processes but Ambari thinks they died and goes red. On further investigation the /mydisk/run/kafka and /mydisk/run/knox are empty so the pid files are not getting created. I re-installed on fresh VM and ran the below to give kafka/knox users access to those dirs before running install wizard but still got same error (even though as those users I can write to those dirs)
chmod +x /mydisk/run
chmod +x /mydisk
Solution:
Since both Kafka and Knox come with bundled bash scripts that manages their process lifecycle amongst other things and we have no control over them, an agreed solution with was to use symlinks.
For example Kafka uses /usr/hdp/current/kafka-broker/bin/kafka bash to start, where values are hardcoded as such
PID_DIR="/var/run/$APP_NAME" on which we have no control over as they are bundled in the Kafka package.
Knox already uses symlinks by default, all we had to do is change said symlink with the ambari managed location.
Diffs (updated)
-----
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py 0170116
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py 74c189e
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py 9e20499
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params.py a04842e
ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py a8b6693
ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 8ffbd67
ambari-server/src/test/python/stacks/2.2/configs/default_custom_path_config.json PRE-CREATION
Diff: https://reviews.apache.org/r/33313/diff/
Testing
-------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [6.912s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.116s]
[INFO] Ambari Web ........................................ SUCCESS [29.655s]
[INFO] Ambari Views ...................................... SUCCESS [1.196s]
[INFO] Ambari Admin View ................................. SUCCESS [19.873s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.116s]
[INFO] Ambari Server ..................................... SUCCESS [2:39.489s]
[INFO] Ambari Agent ...................................... SUCCESS [9.179s]
[INFO] Ambari Client ..................................... SUCCESS [0.202s]
[INFO] Ambari Python Client .............................. SUCCESS [1.121s]
[INFO] Ambari Groovy Client .............................. SUCCESS [2.917s]
[INFO] Ambari Shell ...................................... SUCCESS [0.108s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.637s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [1.760s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:55.047s
[INFO] Finished at: Fri Apr 17 15:31:16 EEST 2015
[INFO] Final Memory: 108M/1125M
[INFO] ------------------------------------------------------------------------
Thanks,
Emil Anca
Re: Review Request 33313: Issues when log/pid dirs are customized -
Kafka and Knox
Posted by Andrew Onischuk <ao...@hortonworks.com>.
> On April 17, 2015, 3:18 p.m., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py, line 88
> > <https://reviews.apache.org/r/33313/diff/1/?file=933832#file933832line88>
> >
> > This break non-root Ambari install
> >
> > We should not native python use tools, please Use Directory("..", action="delete") and so on. Like it's done in other services, that works using sudo.
this includes all other python native function. Like unlink etc os.path.exists etc. they all work w/o sudo, however Ambari should support running under sudoer non-root.
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33313/#review80458
-----------------------------------------------------------
On April 17, 2015, 3:15 p.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33313/
> -----------------------------------------------------------
>
> (Updated April 17, 2015, 3:15 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk and Robert Levas.
>
>
> Bugs: AMBARI-10562
> https://issues.apache.org/jira/browse/AMBARI-10562
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Problem:
>
> Customization of the log and pid dir location using ambari, does not work with Kafka and Knox.
>
>
> After the install wizard error out I also noticed: (all Kafka/Knox pid/log dir paths have a /mydisk/ custom location prefix appended)
>
> * Starting Kafka/Knox from Ambari actually starts the processes but Ambari thinks they died and goes red. On further investigation the /mydisk/run/kafka and /mydisk/run/knox are empty so the pid files are not getting created. I re-installed on fresh VM and ran the below to give kafka/knox users access to those dirs before running install wizard but still got same error (even though as those users I can write to those dirs)
>
> chmod +x /mydisk/run
> chmod +x /mydisk
>
> Solution:
>
> Since both Kafka and Knox come with bundled bash scripts that manages their process lifecycle amongst other things and we have no control over them, an agreed solution with was to use symlinks.
>
> For example Kafka uses /usr/hdp/current/kafka-broker/bin/kafka bash to start, where values are hardcoded as such
> PID_DIR="/var/run/$APP_NAME" on which we have no control over as they are bundled in the Kafka package.
>
> Knox already uses symlinks by default, all we had to do is change said symlink with the ambari managed location.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py 0170116
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py 419639e
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py 9e20499
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params.py 13b0868
> ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 8ffbd67
>
> Diff: https://reviews.apache.org/r/33313/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [6.912s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.116s]
> [INFO] Ambari Web ........................................ SUCCESS [29.655s]
> [INFO] Ambari Views ...................................... SUCCESS [1.196s]
> [INFO] Ambari Admin View ................................. SUCCESS [19.873s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.116s]
> [INFO] Ambari Server ..................................... SUCCESS [2:39.489s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.179s]
> [INFO] Ambari Client ..................................... SUCCESS [0.202s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.121s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.917s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.108s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.637s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.760s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3:55.047s
> [INFO] Finished at: Fri Apr 17 15:31:16 EEST 2015
> [INFO] Final Memory: 108M/1125M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Emil Anca
>
>
Re: Review Request 33313: Issues when log/pid dirs are customized -
Kafka and Knox
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33313/#review80458
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py
<https://reviews.apache.org/r/33313/#comment130371>
This break non-root Ambari install
We should not native python use tools, please Use Directory("..", action="delete") and so on. Like it's done in other services, that works using sudo.
- Andrew Onischuk
On April 17, 2015, 3:15 p.m., Emil Anca wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33313/
> -----------------------------------------------------------
>
> (Updated April 17, 2015, 3:15 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk and Robert Levas.
>
>
> Bugs: AMBARI-10562
> https://issues.apache.org/jira/browse/AMBARI-10562
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Problem:
>
> Customization of the log and pid dir location using ambari, does not work with Kafka and Knox.
>
>
> After the install wizard error out I also noticed: (all Kafka/Knox pid/log dir paths have a /mydisk/ custom location prefix appended)
>
> * Starting Kafka/Knox from Ambari actually starts the processes but Ambari thinks they died and goes red. On further investigation the /mydisk/run/kafka and /mydisk/run/knox are empty so the pid files are not getting created. I re-installed on fresh VM and ran the below to give kafka/knox users access to those dirs before running install wizard but still got same error (even though as those users I can write to those dirs)
>
> chmod +x /mydisk/run
> chmod +x /mydisk
>
> Solution:
>
> Since both Kafka and Knox come with bundled bash scripts that manages their process lifecycle amongst other things and we have no control over them, an agreed solution with was to use symlinks.
>
> For example Kafka uses /usr/hdp/current/kafka-broker/bin/kafka bash to start, where values are hardcoded as such
> PID_DIR="/var/run/$APP_NAME" on which we have no control over as they are bundled in the Kafka package.
>
> Knox already uses symlinks by default, all we had to do is change said symlink with the ambari managed location.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka.py 0170116
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py 419639e
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py 9e20499
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params.py 13b0868
> ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 8ffbd67
>
> Diff: https://reviews.apache.org/r/33313/diff/
>
>
> Testing
> -------
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [6.912s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.116s]
> [INFO] Ambari Web ........................................ SUCCESS [29.655s]
> [INFO] Ambari Views ...................................... SUCCESS [1.196s]
> [INFO] Ambari Admin View ................................. SUCCESS [19.873s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [1.116s]
> [INFO] Ambari Server ..................................... SUCCESS [2:39.489s]
> [INFO] Ambari Agent ...................................... SUCCESS [9.179s]
> [INFO] Ambari Client ..................................... SUCCESS [0.202s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.121s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.917s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.108s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.637s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.760s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 3:55.047s
> [INFO] Finished at: Fri Apr 17 15:31:16 EEST 2015
> [INFO] Final Memory: 108M/1125M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Emil Anca
>
>