You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Nahappan Somasundaram <ns...@hortonworks.com> on 2017/01/31 23:37:29 UTC
Review Request 56142: AMBARI-19806: After setting up hadoop
credential, cannot start Hive Metastore
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56142/
-----------------------------------------------------------
Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sumit Mohanty.
Bugs: AMBARI-19806
https://issues.apache.org/jira/browse/AMBARI-19806
Repository: ambari
Description
-------
AMBARI-19806: After setting up hadoop credential, cannot start Hive Metastore
** Issue **
After setting up hadoop credential, the clear text password is no longer available in the command JSON. The password is required by schematool, DBConnectionVerification and sql (add/remove user) when executing the hive stack scripts. This causes Hive Metastore to fail.
** Fix **
Using a new CLI over the hadoop credential store, get the clear text password from the JCEKS provider when credential store is enabled and use it where required.
Diffs
-----
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 8f1848cb7c212b5613bc6923a6b81c3748c66f06
ambari-server/pom.xml af9bf7ae483776723fc90a3e2571d268d808c751
ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java PRE-CREATION
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 91a337f5ba1574877bce7757f1e53e0d0355d19b
ambari-server/src/test/java/org/apache/ambari/server/credentialapi/CredentialUtilTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/56142/diff/
Testing
-------
** 1. mvn clean install -DskipTests **
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [9.787s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.038s]
[INFO] Ambari Web ........................................ SUCCESS [1:11.600s]
[INFO] Ambari Views ...................................... SUCCESS [1.212s]
[INFO] Ambari Admin View ................................. SUCCESS [9.521s]
[INFO] utility ........................................... SUCCESS [0.281s]
[INFO] ambari-metrics .................................... SUCCESS [0.829s]
[INFO] Ambari Metrics Common ............................. SUCCESS [8.550s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [4.207s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [1.587s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [1.436s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [3.319s]
[INFO] Ambari Metrics Storm Sink (Legacy) ................ SUCCESS [1.613s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [12.426s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [2.481s]
[INFO] Ambari Metrics Grafana ............................ SUCCESS [0.990s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [1:21.512s]
[INFO] Ambari Server ..................................... SUCCESS [5:05.744s]
[INFO] Ambari Functional Tests ........................... SUCCESS [7.163s]
[INFO] Ambari Agent ...................................... SUCCESS [52.341s]
[INFO] Ambari Client ..................................... SUCCESS [0.091s]
[INFO] Ambari Python Client .............................. SUCCESS [1.031s]
[INFO] Ambari Groovy Client .............................. SUCCESS [2.829s]
[INFO] Ambari Shell ...................................... SUCCESS [0.050s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.746s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [1.342s]
[INFO] ambari-logsearch .................................. SUCCESS [0.684s]
[INFO] Ambari Logsearch Appender ......................... SUCCESS [0.342s]
[INFO] Ambari Logsearch Portal ........................... SUCCESS [13.636s]
[INFO] Ambari Logsearch Log Feeder ....................... SUCCESS [6.913s]
[INFO] Ambari Logsearch Solr Client ...................... SUCCESS [2.166s]
[INFO] Ambari Logsearch Assembly ......................... SUCCESS [0.186s]
[INFO] Ambari Logsearch Integration Test ................. SUCCESS [0.503s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:21.649s
[INFO] Finished at: Tue Jan 31 15:05:49 PST 2017
[INFO] Final Memory: 338M/1123M
[INFO] ------------------------------------------------------------------------
** 2. mvn test -DskipPythonTests -Dtest=CredentialUtilTest **
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 5320 licence.
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
[INFO] skipping execute as per configuraion
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
[INFO] Starting audit...
Audit done.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30.484s
[INFO] Finished at: Tue Jan 31 15:09:13 PST 2017
[INFO] Final Memory: 58M/796M
[INFO] ------------------------------------------------------------------------
** 3. mvn test -DskipSurefireTests **
----------------------------------------------------------------------
Ran 270 tests in 6.740s
OK
----------------------------------------------------------------------
Total run:1172
Total errors:0
Total failures:0
OK
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
[INFO] Starting audit...
Audit done.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:19.072s
[INFO] Finished at: Tue Jan 31 15:33:59 PST 2017
[INFO] Final Memory: 69M/786M
[INFO] ------------------------------------------------------------------------
** 4. New UTs **
Added CredentialUtilTest suite of tests to test the CredentialUtil CLI.
** 5. Manual tests **
Deployed the local build and modified scripts to a VM and tested the changes on HIVE service with credential store enabled. Verified that HIVE could be stopped, started or restarted.
Thanks,
Nahappan Somasundaram
Re: Review Request 56142: AMBARI-19806: After setting up hadoop
credential, cannot start Hive Metastore
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56142/#review163857
-----------------------------------------------------------
Ship it!
Ship It!
- Sumit Mohanty
On Feb. 1, 2017, 7:02 p.m., Nahappan Somasundaram wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56142/
> -----------------------------------------------------------
>
> (Updated Feb. 1, 2017, 7:02 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19806
> https://issues.apache.org/jira/browse/AMBARI-19806
>
>
> Repository: ambari
>
>
> Description
> -------
>
> AMBARI-19806: After setting up hadoop credential, cannot start Hive Metastore
>
> ** Issue **
> After setting up hadoop credential, the clear text password is no longer available in the command JSON. The password is required by schematool, DBConnectionVerification and sql (add/remove user) when executing the hive stack scripts. This causes Hive Metastore to fail.
>
> ** Fix **
> Using a new CLI over the hadoop credential store, get the clear text password from the JCEKS provider when credential store is enabled and use it where required.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 8f1848cb7c212b5613bc6923a6b81c3748c66f06
> ambari-server/pom.xml af9bf7ae483776723fc90a3e2571d268d808c751
> ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 91a337f5ba1574877bce7757f1e53e0d0355d19b
> ambari-server/src/test/java/org/apache/ambari/server/credentialapi/CredentialUtilTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56142/diff/
>
>
> Testing
> -------
>
> ** 1. mvn clean install -DskipTests **
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [9.787s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.038s]
> [INFO] Ambari Web ........................................ SUCCESS [1:11.600s]
> [INFO] Ambari Views ...................................... SUCCESS [1.212s]
> [INFO] Ambari Admin View ................................. SUCCESS [9.521s]
> [INFO] utility ........................................... SUCCESS [0.281s]
> [INFO] ambari-metrics .................................... SUCCESS [0.829s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [8.550s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [4.207s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [1.587s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [1.436s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [3.319s]
> [INFO] Ambari Metrics Storm Sink (Legacy) ................ SUCCESS [1.613s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [12.426s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [2.481s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [0.990s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:21.512s]
> [INFO] Ambari Server ..................................... SUCCESS [5:05.744s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [7.163s]
> [INFO] Ambari Agent ...................................... SUCCESS [52.341s]
> [INFO] Ambari Client ..................................... SUCCESS [0.091s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.031s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.829s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.050s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.746s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.342s]
> [INFO] ambari-logsearch .................................. SUCCESS [0.684s]
> [INFO] Ambari Logsearch Appender ......................... SUCCESS [0.342s]
> [INFO] Ambari Logsearch Portal ........................... SUCCESS [13.636s]
> [INFO] Ambari Logsearch Log Feeder ....................... SUCCESS [6.913s]
> [INFO] Ambari Logsearch Solr Client ...................... SUCCESS [2.166s]
> [INFO] Ambari Logsearch Assembly ......................... SUCCESS [0.186s]
> [INFO] Ambari Logsearch Integration Test ................. SUCCESS [0.503s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 10:21.649s
> [INFO] Finished at: Tue Jan 31 15:05:49 PST 2017
> [INFO] Final Memory: 338M/1123M
> [INFO] ------------------------------------------------------------------------
>
> ** 2. mvn test -DskipPythonTests -Dtest=CredentialUtilTest **
>
> [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 5320 licence.
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
> [INFO] skipping execute as per configuraion
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
> [INFO] Starting audit...
> Audit done.
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30.484s
> [INFO] Finished at: Tue Jan 31 15:09:13 PST 2017
> [INFO] Final Memory: 58M/796M
> [INFO] ------------------------------------------------------------------------
>
> ** 3. mvn test -DskipSurefireTests **
> ----------------------------------------------------------------------
> Ran 270 tests in 6.740s
>
> OK
> ----------------------------------------------------------------------
> Total run:1172
> Total errors:0
> Total failures:0
> OK
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
> [INFO] Starting audit...
> Audit done.
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:19.072s
> [INFO] Finished at: Tue Jan 31 15:33:59 PST 2017
> [INFO] Final Memory: 69M/786M
> [INFO] ------------------------------------------------------------------------
>
> ** 4. New UTs **
> Added CredentialUtilTest suite of tests to test the CredentialUtil CLI.
>
> ** 5. Manual tests **
> Deployed the local build and modified scripts to a VM and tested the changes on HIVE service with credential store enabled. Verified that HIVE could be stopped, started or restarted.
>
>
> Thanks,
>
> Nahappan Somasundaram
>
>
Re: Review Request 56142: AMBARI-19806: After setting up hadoop
credential, cannot start Hive Metastore
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56142/#review163858
-----------------------------------------------------------
Ship it!
Ship It!
- Robert Levas
On Feb. 1, 2017, 2:02 p.m., Nahappan Somasundaram wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56142/
> -----------------------------------------------------------
>
> (Updated Feb. 1, 2017, 2:02 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19806
> https://issues.apache.org/jira/browse/AMBARI-19806
>
>
> Repository: ambari
>
>
> Description
> -------
>
> AMBARI-19806: After setting up hadoop credential, cannot start Hive Metastore
>
> ** Issue **
> After setting up hadoop credential, the clear text password is no longer available in the command JSON. The password is required by schematool, DBConnectionVerification and sql (add/remove user) when executing the hive stack scripts. This causes Hive Metastore to fail.
>
> ** Fix **
> Using a new CLI over the hadoop credential store, get the clear text password from the JCEKS provider when credential store is enabled and use it where required.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 8f1848cb7c212b5613bc6923a6b81c3748c66f06
> ambari-server/pom.xml af9bf7ae483776723fc90a3e2571d268d808c751
> ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 91a337f5ba1574877bce7757f1e53e0d0355d19b
> ambari-server/src/test/java/org/apache/ambari/server/credentialapi/CredentialUtilTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56142/diff/
>
>
> Testing
> -------
>
> ** 1. mvn clean install -DskipTests **
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [9.787s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.038s]
> [INFO] Ambari Web ........................................ SUCCESS [1:11.600s]
> [INFO] Ambari Views ...................................... SUCCESS [1.212s]
> [INFO] Ambari Admin View ................................. SUCCESS [9.521s]
> [INFO] utility ........................................... SUCCESS [0.281s]
> [INFO] ambari-metrics .................................... SUCCESS [0.829s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [8.550s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [4.207s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [1.587s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [1.436s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [3.319s]
> [INFO] Ambari Metrics Storm Sink (Legacy) ................ SUCCESS [1.613s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [12.426s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [2.481s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [0.990s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:21.512s]
> [INFO] Ambari Server ..................................... SUCCESS [5:05.744s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [7.163s]
> [INFO] Ambari Agent ...................................... SUCCESS [52.341s]
> [INFO] Ambari Client ..................................... SUCCESS [0.091s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.031s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.829s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.050s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.746s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.342s]
> [INFO] ambari-logsearch .................................. SUCCESS [0.684s]
> [INFO] Ambari Logsearch Appender ......................... SUCCESS [0.342s]
> [INFO] Ambari Logsearch Portal ........................... SUCCESS [13.636s]
> [INFO] Ambari Logsearch Log Feeder ....................... SUCCESS [6.913s]
> [INFO] Ambari Logsearch Solr Client ...................... SUCCESS [2.166s]
> [INFO] Ambari Logsearch Assembly ......................... SUCCESS [0.186s]
> [INFO] Ambari Logsearch Integration Test ................. SUCCESS [0.503s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 10:21.649s
> [INFO] Finished at: Tue Jan 31 15:05:49 PST 2017
> [INFO] Final Memory: 338M/1123M
> [INFO] ------------------------------------------------------------------------
>
> ** 2. mvn test -DskipPythonTests -Dtest=CredentialUtilTest **
>
> [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 5320 licence.
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
> [INFO] skipping execute as per configuraion
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
> [INFO] Starting audit...
> Audit done.
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30.484s
> [INFO] Finished at: Tue Jan 31 15:09:13 PST 2017
> [INFO] Final Memory: 58M/796M
> [INFO] ------------------------------------------------------------------------
>
> ** 3. mvn test -DskipSurefireTests **
> ----------------------------------------------------------------------
> Ran 270 tests in 6.740s
>
> OK
> ----------------------------------------------------------------------
> Total run:1172
> Total errors:0
> Total failures:0
> OK
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
> [INFO] Starting audit...
> Audit done.
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:19.072s
> [INFO] Finished at: Tue Jan 31 15:33:59 PST 2017
> [INFO] Final Memory: 69M/786M
> [INFO] ------------------------------------------------------------------------
>
> ** 4. New UTs **
> Added CredentialUtilTest suite of tests to test the CredentialUtil CLI.
>
> ** 5. Manual tests **
> Deployed the local build and modified scripts to a VM and tested the changes on HIVE service with credential store enabled. Verified that HIVE could be stopped, started or restarted.
>
>
> Thanks,
>
> Nahappan Somasundaram
>
>
Re: Review Request 56142: AMBARI-19806: After setting up hadoop
credential, cannot start Hive Metastore
Posted by Nahappan Somasundaram <ns...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56142/
-----------------------------------------------------------
(Updated Feb. 1, 2017, 11:02 a.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sumit Mohanty.
Changes
-------
Added javadoc to CredentialUtil class. Also fixed tabbing issues with comments.
Bugs: AMBARI-19806
https://issues.apache.org/jira/browse/AMBARI-19806
Repository: ambari
Description
-------
AMBARI-19806: After setting up hadoop credential, cannot start Hive Metastore
** Issue **
After setting up hadoop credential, the clear text password is no longer available in the command JSON. The password is required by schematool, DBConnectionVerification and sql (add/remove user) when executing the hive stack scripts. This causes Hive Metastore to fail.
** Fix **
Using a new CLI over the hadoop credential store, get the clear text password from the JCEKS provider when credential store is enabled and use it where required.
Diffs (updated)
-----
ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 8f1848cb7c212b5613bc6923a6b81c3748c66f06
ambari-server/pom.xml af9bf7ae483776723fc90a3e2571d268d808c751
ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java PRE-CREATION
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 91a337f5ba1574877bce7757f1e53e0d0355d19b
ambari-server/src/test/java/org/apache/ambari/server/credentialapi/CredentialUtilTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/56142/diff/
Testing
-------
** 1. mvn clean install -DskipTests **
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [9.787s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.038s]
[INFO] Ambari Web ........................................ SUCCESS [1:11.600s]
[INFO] Ambari Views ...................................... SUCCESS [1.212s]
[INFO] Ambari Admin View ................................. SUCCESS [9.521s]
[INFO] utility ........................................... SUCCESS [0.281s]
[INFO] ambari-metrics .................................... SUCCESS [0.829s]
[INFO] Ambari Metrics Common ............................. SUCCESS [8.550s]
[INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [4.207s]
[INFO] Ambari Metrics Flume Sink ......................... SUCCESS [1.587s]
[INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [1.436s]
[INFO] Ambari Metrics Storm Sink ......................... SUCCESS [3.319s]
[INFO] Ambari Metrics Storm Sink (Legacy) ................ SUCCESS [1.613s]
[INFO] Ambari Metrics Collector .......................... SUCCESS [12.426s]
[INFO] Ambari Metrics Monitor ............................ SUCCESS [2.481s]
[INFO] Ambari Metrics Grafana ............................ SUCCESS [0.990s]
[INFO] Ambari Metrics Assembly ........................... SUCCESS [1:21.512s]
[INFO] Ambari Server ..................................... SUCCESS [5:05.744s]
[INFO] Ambari Functional Tests ........................... SUCCESS [7.163s]
[INFO] Ambari Agent ...................................... SUCCESS [52.341s]
[INFO] Ambari Client ..................................... SUCCESS [0.091s]
[INFO] Ambari Python Client .............................. SUCCESS [1.031s]
[INFO] Ambari Groovy Client .............................. SUCCESS [2.829s]
[INFO] Ambari Shell ...................................... SUCCESS [0.050s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.746s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [1.342s]
[INFO] ambari-logsearch .................................. SUCCESS [0.684s]
[INFO] Ambari Logsearch Appender ......................... SUCCESS [0.342s]
[INFO] Ambari Logsearch Portal ........................... SUCCESS [13.636s]
[INFO] Ambari Logsearch Log Feeder ....................... SUCCESS [6.913s]
[INFO] Ambari Logsearch Solr Client ...................... SUCCESS [2.166s]
[INFO] Ambari Logsearch Assembly ......................... SUCCESS [0.186s]
[INFO] Ambari Logsearch Integration Test ................. SUCCESS [0.503s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:21.649s
[INFO] Finished at: Tue Jan 31 15:05:49 PST 2017
[INFO] Final Memory: 338M/1123M
[INFO] ------------------------------------------------------------------------
** 2. mvn test -DskipPythonTests -Dtest=CredentialUtilTest **
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 5320 licence.
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
[INFO] skipping execute as per configuraion
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
[INFO] Starting audit...
Audit done.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30.484s
[INFO] Finished at: Tue Jan 31 15:09:13 PST 2017
[INFO] Final Memory: 58M/796M
[INFO] ------------------------------------------------------------------------
** 3. mvn test -DskipSurefireTests **
----------------------------------------------------------------------
Ran 270 tests in 6.740s
OK
----------------------------------------------------------------------
Total run:1172
Total errors:0
Total failures:0
OK
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
[INFO] Starting audit...
Audit done.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:19.072s
[INFO] Finished at: Tue Jan 31 15:33:59 PST 2017
[INFO] Final Memory: 69M/786M
[INFO] ------------------------------------------------------------------------
** 4. New UTs **
Added CredentialUtilTest suite of tests to test the CredentialUtil CLI.
** 5. Manual tests **
Deployed the local build and modified scripts to a VM and tested the changes on HIVE service with credential store enabled. Verified that HIVE could be stopped, started or restarted.
Thanks,
Nahappan Somasundaram
Re: Review Request 56142: AMBARI-19806: After setting up hadoop
credential, cannot start Hive Metastore
Posted by Nahappan Somasundaram <ns...@hortonworks.com>.
> On Feb. 1, 2017, 8:39 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java, line 236
> > <https://reviews.apache.org/r/56142/diff/1/?file=1620823#file1620823line236>
> >
> > Does this cause an issue if the path to the file contains uppercase characters?
jceksPath is a temporary variable, used for figuring out if the path starts with a specific pattern. The path specified by the user is not lowercased. It is used as is or prefixed with jceks://.
> On Feb. 1, 2017, 8:39 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java, lines 420-423
> > <https://reviews.apache.org/r/56142/diff/1/?file=1620823#file1620823line420>
> >
> > This changes the "global" value of the varaible. It is ok if `overwrite` isn't used again for a different purpose. However maybe the tool, is can only handle one operation at a time and this is not an issue.
Yes, it is used only once even though it is a global variable, since the tool can execute only one command at a time.
- Nahappan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56142/#review163824
-----------------------------------------------------------
On Jan. 31, 2017, 3:37 p.m., Nahappan Somasundaram wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56142/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 3:37 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19806
> https://issues.apache.org/jira/browse/AMBARI-19806
>
>
> Repository: ambari
>
>
> Description
> -------
>
> AMBARI-19806: After setting up hadoop credential, cannot start Hive Metastore
>
> ** Issue **
> After setting up hadoop credential, the clear text password is no longer available in the command JSON. The password is required by schematool, DBConnectionVerification and sql (add/remove user) when executing the hive stack scripts. This causes Hive Metastore to fail.
>
> ** Fix **
> Using a new CLI over the hadoop credential store, get the clear text password from the JCEKS provider when credential store is enabled and use it where required.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 8f1848cb7c212b5613bc6923a6b81c3748c66f06
> ambari-server/pom.xml af9bf7ae483776723fc90a3e2571d268d808c751
> ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 91a337f5ba1574877bce7757f1e53e0d0355d19b
> ambari-server/src/test/java/org/apache/ambari/server/credentialapi/CredentialUtilTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56142/diff/
>
>
> Testing
> -------
>
> ** 1. mvn clean install -DskipTests **
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [9.787s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.038s]
> [INFO] Ambari Web ........................................ SUCCESS [1:11.600s]
> [INFO] Ambari Views ...................................... SUCCESS [1.212s]
> [INFO] Ambari Admin View ................................. SUCCESS [9.521s]
> [INFO] utility ........................................... SUCCESS [0.281s]
> [INFO] ambari-metrics .................................... SUCCESS [0.829s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [8.550s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [4.207s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [1.587s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [1.436s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [3.319s]
> [INFO] Ambari Metrics Storm Sink (Legacy) ................ SUCCESS [1.613s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [12.426s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [2.481s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [0.990s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:21.512s]
> [INFO] Ambari Server ..................................... SUCCESS [5:05.744s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [7.163s]
> [INFO] Ambari Agent ...................................... SUCCESS [52.341s]
> [INFO] Ambari Client ..................................... SUCCESS [0.091s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.031s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.829s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.050s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.746s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.342s]
> [INFO] ambari-logsearch .................................. SUCCESS [0.684s]
> [INFO] Ambari Logsearch Appender ......................... SUCCESS [0.342s]
> [INFO] Ambari Logsearch Portal ........................... SUCCESS [13.636s]
> [INFO] Ambari Logsearch Log Feeder ....................... SUCCESS [6.913s]
> [INFO] Ambari Logsearch Solr Client ...................... SUCCESS [2.166s]
> [INFO] Ambari Logsearch Assembly ......................... SUCCESS [0.186s]
> [INFO] Ambari Logsearch Integration Test ................. SUCCESS [0.503s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 10:21.649s
> [INFO] Finished at: Tue Jan 31 15:05:49 PST 2017
> [INFO] Final Memory: 338M/1123M
> [INFO] ------------------------------------------------------------------------
>
> ** 2. mvn test -DskipPythonTests -Dtest=CredentialUtilTest **
>
> [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 5320 licence.
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
> [INFO] skipping execute as per configuraion
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
> [INFO] Starting audit...
> Audit done.
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30.484s
> [INFO] Finished at: Tue Jan 31 15:09:13 PST 2017
> [INFO] Final Memory: 58M/796M
> [INFO] ------------------------------------------------------------------------
>
> ** 3. mvn test -DskipSurefireTests **
> ----------------------------------------------------------------------
> Ran 270 tests in 6.740s
>
> OK
> ----------------------------------------------------------------------
> Total run:1172
> Total errors:0
> Total failures:0
> OK
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
> [INFO] Starting audit...
> Audit done.
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:19.072s
> [INFO] Finished at: Tue Jan 31 15:33:59 PST 2017
> [INFO] Final Memory: 69M/786M
> [INFO] ------------------------------------------------------------------------
>
> ** 4. New UTs **
> Added CredentialUtilTest suite of tests to test the CredentialUtil CLI.
>
> ** 5. Manual tests **
> Deployed the local build and modified scripts to a VM and tested the changes on HIVE service with credential store enabled. Verified that HIVE could be stopped, started or restarted.
>
>
> Thanks,
>
> Nahappan Somasundaram
>
>
Re: Review Request 56142: AMBARI-19806: After setting up hadoop
credential, cannot start Hive Metastore
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56142/#review163824
-----------------------------------------------------------
Fix it, then Ship it!
ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java (line 38)
<https://reviews.apache.org/r/56142/#comment235315>
Missing JavaDoc
ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java (line 236)
<https://reviews.apache.org/r/56142/#comment235316>
Does this cause an issue if the path to the file contains uppercase characters?
ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java (lines 420 - 423)
<https://reviews.apache.org/r/56142/#comment235318>
This changes the "global" value of the varaible. It is ok if `overwrite` isn't used again for a different purpose. However maybe the tool, is can only handle one operation at a time and this is not an issue.
- Robert Levas
On Jan. 31, 2017, 6:37 p.m., Nahappan Somasundaram wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56142/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 6:37 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19806
> https://issues.apache.org/jira/browse/AMBARI-19806
>
>
> Repository: ambari
>
>
> Description
> -------
>
> AMBARI-19806: After setting up hadoop credential, cannot start Hive Metastore
>
> ** Issue **
> After setting up hadoop credential, the clear text password is no longer available in the command JSON. The password is required by schematool, DBConnectionVerification and sql (add/remove user) when executing the hive stack scripts. This causes Hive Metastore to fail.
>
> ** Fix **
> Using a new CLI over the hadoop credential store, get the clear text password from the JCEKS provider when credential store is enabled and use it where required.
>
>
> Diffs
> -----
>
> ambari-agent/src/main/python/ambari_agent/CustomServiceOrchestrator.py 8f1848cb7c212b5613bc6923a6b81c3748c66f06
> ambari-server/pom.xml af9bf7ae483776723fc90a3e2571d268d808c751
> ambari-server/src/main/java/org/apache/ambari/server/credentialapi/CredentialUtil.java PRE-CREATION
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 91a337f5ba1574877bce7757f1e53e0d0355d19b
> ambari-server/src/test/java/org/apache/ambari/server/credentialapi/CredentialUtilTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56142/diff/
>
>
> Testing
> -------
>
> ** 1. mvn clean install -DskipTests **
>
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ....................................... SUCCESS [9.787s]
> [INFO] Apache Ambari Project POM ......................... SUCCESS [0.038s]
> [INFO] Ambari Web ........................................ SUCCESS [1:11.600s]
> [INFO] Ambari Views ...................................... SUCCESS [1.212s]
> [INFO] Ambari Admin View ................................. SUCCESS [9.521s]
> [INFO] utility ........................................... SUCCESS [0.281s]
> [INFO] ambari-metrics .................................... SUCCESS [0.829s]
> [INFO] Ambari Metrics Common ............................. SUCCESS [8.550s]
> [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [4.207s]
> [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [1.587s]
> [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [1.436s]
> [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [3.319s]
> [INFO] Ambari Metrics Storm Sink (Legacy) ................ SUCCESS [1.613s]
> [INFO] Ambari Metrics Collector .......................... SUCCESS [12.426s]
> [INFO] Ambari Metrics Monitor ............................ SUCCESS [2.481s]
> [INFO] Ambari Metrics Grafana ............................ SUCCESS [0.990s]
> [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:21.512s]
> [INFO] Ambari Server ..................................... SUCCESS [5:05.744s]
> [INFO] Ambari Functional Tests ........................... SUCCESS [7.163s]
> [INFO] Ambari Agent ...................................... SUCCESS [52.341s]
> [INFO] Ambari Client ..................................... SUCCESS [0.091s]
> [INFO] Ambari Python Client .............................. SUCCESS [1.031s]
> [INFO] Ambari Groovy Client .............................. SUCCESS [2.829s]
> [INFO] Ambari Shell ...................................... SUCCESS [0.050s]
> [INFO] Ambari Python Shell ............................... SUCCESS [0.746s]
> [INFO] Ambari Groovy Shell ............................... SUCCESS [1.342s]
> [INFO] ambari-logsearch .................................. SUCCESS [0.684s]
> [INFO] Ambari Logsearch Appender ......................... SUCCESS [0.342s]
> [INFO] Ambari Logsearch Portal ........................... SUCCESS [13.636s]
> [INFO] Ambari Logsearch Log Feeder ....................... SUCCESS [6.913s]
> [INFO] Ambari Logsearch Solr Client ...................... SUCCESS [2.166s]
> [INFO] Ambari Logsearch Assembly ......................... SUCCESS [0.186s]
> [INFO] Ambari Logsearch Integration Test ................. SUCCESS [0.503s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 10:21.649s
> [INFO] Finished at: Tue Jan 31 15:05:49 PST 2017
> [INFO] Final Memory: 338M/1123M
> [INFO] ------------------------------------------------------------------------
>
> ** 2. mvn test -DskipPythonTests -Dtest=CredentialUtilTest **
>
> [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 5320 licence.
> [INFO]
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server ---
> [INFO] skipping execute as per configuraion
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
> [INFO] Starting audit...
> Audit done.
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 30.484s
> [INFO] Finished at: Tue Jan 31 15:09:13 PST 2017
> [INFO] Final Memory: 58M/796M
> [INFO] ------------------------------------------------------------------------
>
> ** 3. mvn test -DskipSurefireTests **
> ----------------------------------------------------------------------
> Ran 270 tests in 6.740s
>
> OK
> ----------------------------------------------------------------------
> Total run:1172
> Total errors:0
> Total failures:0
> OK
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ ambari-server ---
> [INFO] Starting audit...
> Audit done.
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:19.072s
> [INFO] Finished at: Tue Jan 31 15:33:59 PST 2017
> [INFO] Final Memory: 69M/786M
> [INFO] ------------------------------------------------------------------------
>
> ** 4. New UTs **
> Added CredentialUtilTest suite of tests to test the CredentialUtil CLI.
>
> ** 5. Manual tests **
> Deployed the local build and modified scripts to a VM and tested the changes on HIVE service with credential store enabled. Verified that HIVE could be stopped, started or restarted.
>
>
> Thanks,
>
> Nahappan Somasundaram
>
>