You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2016/04/19 05:09:05 UTC
Review Request 46363: Kerberos wizard stuck trying to schedule
service check operation
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46363/
-----------------------------------------------------------
Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Sumit Mohanty, Srimanth Gunturi, and Sid Wagle.
Bugs: AMBARI-15961
https://issues.apache.org/jira/browse/AMBARI-15961
Repository: ambari
Description
-------
Attached jstack after firing up the curl call.
```
curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST -d'{"RequestInfo":{"context":"Kerberos Service Check","command":"KERBEROS_SERVICE_CHECK","operation_level":{"level":"CLUSTER","cluster_name":"c1"}},"Requests/resource_filters":[{"service_name":"KERBEROS"}]}' http://104.196.89.51:8080/api/v1/clusters/c1/requests
```
Behavior:
- Call timedout on the UI and wizard cannot proceed further.
- Exception in the server log after long wait:
```
12 Apr 2016 23:01:45,231 INFO [qtp-ambari-client-818] AmbariManagementControllerImpl:3376 - Received action execution request, clusterName=c1, request=isCommand :true, action :null, command :KERBEROS_SERVICE_CHECK, inputs :{}, resourceFilters: [RequestResourceFilter{serviceName='KERBEROS', componentName='null', hostNames=[]}], exclusive: false, clusterName :c1
12 Apr 2016 23:01:45,409 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HIVE
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: AMBARI_METRICS
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HDFS
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: MAPREDUCE2
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: OOZIE
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: TEZ
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HBASE
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: ZOOKEEPER
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: KERBEROS
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: YARN
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: PIG
12 Apr 2016 23:03:49,984 WARN [qtp-ambari-client-818] MITKerberosOperationHandler:434 - Failed to execute kadmin:
Command: /usr/bin/kadmin -s perf-b-3.c.pramod-thangali.internal -p admin -w ******** -r EXAMPLE.COM -q "get_principal admin"
ExitCode: 1
STDOUT: Authenticating as principal admin with password.
STDERR: kadmin: Client not found in Kerberos database while initializing kadmin interface
```
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 737b0aa
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ADKerberosOperationHandler.java a1e1544
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreatePrincipalsServerAction.java 55e046e
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DestroyPrincipalsServerAction.java 93daae8
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosMissingAdminCredentialsException.java ddb2769
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandler.java e3d31de
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalAlreadyExistsException.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalDoesNotExistException.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java 90858fb
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 34de6a8
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java eefaf04
Diff: https://reviews.apache.org/r/46363/diff/
Testing
-------
Manually tested in large cluster against MIT KDC and small cluster against Active Directory
# Local test results:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:04:20.973s
[INFO] Finished at: Mon Apr 18 21:36:25 EDT 2016
[INFO] Final Memory: 71M/1554M
[INFO] ------------------------------------------------------------------------
# Jenkins test results: PENDING
Thanks,
Robert Levas
Re: Review Request 46363: Kerberos wizard stuck trying to schedule
service check operation
Posted by Oliver Szabo <os...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46363/#review129539
-----------------------------------------------------------
Ship it!
Ship It!
- Oliver Szabo
On April 19, 2016, 3:12 a.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46363/
> -----------------------------------------------------------
>
> (Updated April 19, 2016, 3:12 a.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Sumit Mohanty, Srimanth Gunturi, and Sid Wagle.
>
>
> Bugs: AMBARI-15961
> https://issues.apache.org/jira/browse/AMBARI-15961
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Attached jstack after firing up the curl call.
>
> ```
> curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST -d'{"RequestInfo":{"context":"Kerberos Service Check","command":"KERBEROS_SERVICE_CHECK","operation_level":{"level":"CLUSTER","cluster_name":"c1"}},"Requests/resource_filters":[{"service_name":"KERBEROS"}]}' http://104.196.89.51:8080/api/v1/clusters/c1/requests
> ```
>
> Behavior:
> - Call timedout on the UI and wizard cannot proceed further.
> - Exception in the server log after long wait:
> ```
> 12 Apr 2016 23:01:45,231 INFO [qtp-ambari-client-818] AmbariManagementControllerImpl:3376 - Received action execution request, clusterName=c1, request=isCommand :true, action :null, command :KERBEROS_SERVICE_CHECK, inputs :{}, resourceFilters: [RequestResourceFilter{serviceName='KERBEROS', componentName='null', hostNames=[]}], exclusive: false, clusterName :c1
> 12 Apr 2016 23:01:45,409 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HIVE
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: AMBARI_METRICS
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HDFS
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: MAPREDUCE2
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: OOZIE
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: TEZ
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HBASE
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: ZOOKEEPER
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: KERBEROS
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: YARN
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: PIG
> 12 Apr 2016 23:03:49,984 WARN [qtp-ambari-client-818] MITKerberosOperationHandler:434 - Failed to execute kadmin:
> Command: /usr/bin/kadmin -s perf-b-3.c.pramod-thangali.internal -p admin -w ******** -r EXAMPLE.COM -q "get_principal admin"
> ExitCode: 1
> STDOUT: Authenticating as principal admin with password.
>
> STDERR: kadmin: Client not found in Kerberos database while initializing kadmin interface
> ```
>
> # Solution
> - Optimize preparation logic to create principal for Kerberos Service Check
> - Optimize logic to create principals in MIT KDC to reduce the time in the Create Principal stage by about 30%
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 737b0aa
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ADKerberosOperationHandler.java a1e1544
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreatePrincipalsServerAction.java 55e046e
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DestroyPrincipalsServerAction.java 93daae8
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosMissingAdminCredentialsException.java ddb2769
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandler.java e3d31de
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalAlreadyExistsException.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalDoesNotExistException.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java 90858fb
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 34de6a8
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java eefaf04
>
> Diff: https://reviews.apache.org/r/46363/diff/
>
>
> Testing
> -------
>
> Manually tested in large cluster against MIT KDC and small cluster against Active Directory
>
> # Local test results:
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:04:20.973s
> [INFO] Finished at: Mon Apr 18 21:36:25 EDT 2016
> [INFO] Final Memory: 71M/1554M
> [INFO] ------------------------------------------------------------------------
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 46363: Kerberos wizard stuck trying to schedule
service check operation
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46363/#review129537
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On April 18, 2016, 11:12 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46363/
> -----------------------------------------------------------
>
> (Updated April 18, 2016, 11:12 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Sumit Mohanty, Srimanth Gunturi, and Sid Wagle.
>
>
> Bugs: AMBARI-15961
> https://issues.apache.org/jira/browse/AMBARI-15961
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Attached jstack after firing up the curl call.
>
> ```
> curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST -d'{"RequestInfo":{"context":"Kerberos Service Check","command":"KERBEROS_SERVICE_CHECK","operation_level":{"level":"CLUSTER","cluster_name":"c1"}},"Requests/resource_filters":[{"service_name":"KERBEROS"}]}' http://104.196.89.51:8080/api/v1/clusters/c1/requests
> ```
>
> Behavior:
> - Call timedout on the UI and wizard cannot proceed further.
> - Exception in the server log after long wait:
> ```
> 12 Apr 2016 23:01:45,231 INFO [qtp-ambari-client-818] AmbariManagementControllerImpl:3376 - Received action execution request, clusterName=c1, request=isCommand :true, action :null, command :KERBEROS_SERVICE_CHECK, inputs :{}, resourceFilters: [RequestResourceFilter{serviceName='KERBEROS', componentName='null', hostNames=[]}], exclusive: false, clusterName :c1
> 12 Apr 2016 23:01:45,409 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HIVE
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: AMBARI_METRICS
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HDFS
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: MAPREDUCE2
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: OOZIE
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: TEZ
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HBASE
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: ZOOKEEPER
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: KERBEROS
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: YARN
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: PIG
> 12 Apr 2016 23:03:49,984 WARN [qtp-ambari-client-818] MITKerberosOperationHandler:434 - Failed to execute kadmin:
> Command: /usr/bin/kadmin -s perf-b-3.c.pramod-thangali.internal -p admin -w ******** -r EXAMPLE.COM -q "get_principal admin"
> ExitCode: 1
> STDOUT: Authenticating as principal admin with password.
>
> STDERR: kadmin: Client not found in Kerberos database while initializing kadmin interface
> ```
>
> # Solution
> - Optimize preparation logic to create principal for Kerberos Service Check
> - Optimize logic to create principals in MIT KDC to reduce the time in the Create Principal stage by about 30%
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 737b0aa
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ADKerberosOperationHandler.java a1e1544
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreatePrincipalsServerAction.java 55e046e
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DestroyPrincipalsServerAction.java 93daae8
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosMissingAdminCredentialsException.java ddb2769
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandler.java e3d31de
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalAlreadyExistsException.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalDoesNotExistException.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java 90858fb
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 34de6a8
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java eefaf04
>
> Diff: https://reviews.apache.org/r/46363/diff/
>
>
> Testing
> -------
>
> Manually tested in large cluster against MIT KDC and small cluster against Active Directory
>
> # Local test results:
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:04:20.973s
> [INFO] Finished at: Mon Apr 18 21:36:25 EDT 2016
> [INFO] Final Memory: 71M/1554M
> [INFO] ------------------------------------------------------------------------
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 46363: Kerberos wizard stuck trying to schedule
service check operation
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46363/#review129518
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On April 18, 2016, 11:12 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46363/
> -----------------------------------------------------------
>
> (Updated April 18, 2016, 11:12 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Sumit Mohanty, Srimanth Gunturi, and Sid Wagle.
>
>
> Bugs: AMBARI-15961
> https://issues.apache.org/jira/browse/AMBARI-15961
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Attached jstack after firing up the curl call.
>
> ```
> curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST -d'{"RequestInfo":{"context":"Kerberos Service Check","command":"KERBEROS_SERVICE_CHECK","operation_level":{"level":"CLUSTER","cluster_name":"c1"}},"Requests/resource_filters":[{"service_name":"KERBEROS"}]}' http://104.196.89.51:8080/api/v1/clusters/c1/requests
> ```
>
> Behavior:
> - Call timedout on the UI and wizard cannot proceed further.
> - Exception in the server log after long wait:
> ```
> 12 Apr 2016 23:01:45,231 INFO [qtp-ambari-client-818] AmbariManagementControllerImpl:3376 - Received action execution request, clusterName=c1, request=isCommand :true, action :null, command :KERBEROS_SERVICE_CHECK, inputs :{}, resourceFilters: [RequestResourceFilter{serviceName='KERBEROS', componentName='null', hostNames=[]}], exclusive: false, clusterName :c1
> 12 Apr 2016 23:01:45,409 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HIVE
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: AMBARI_METRICS
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HDFS
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: MAPREDUCE2
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: OOZIE
> 12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: TEZ
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HBASE
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: ZOOKEEPER
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: KERBEROS
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: YARN
> 12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: PIG
> 12 Apr 2016 23:03:49,984 WARN [qtp-ambari-client-818] MITKerberosOperationHandler:434 - Failed to execute kadmin:
> Command: /usr/bin/kadmin -s perf-b-3.c.pramod-thangali.internal -p admin -w ******** -r EXAMPLE.COM -q "get_principal admin"
> ExitCode: 1
> STDOUT: Authenticating as principal admin with password.
>
> STDERR: kadmin: Client not found in Kerberos database while initializing kadmin interface
> ```
>
> # Solution
> - Optimize preparation logic to create principal for Kerberos Service Check
> - Optimize logic to create principals in MIT KDC to reduce the time in the Create Principal stage by about 30%
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 737b0aa
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ADKerberosOperationHandler.java a1e1544
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreatePrincipalsServerAction.java 55e046e
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DestroyPrincipalsServerAction.java 93daae8
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosMissingAdminCredentialsException.java ddb2769
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandler.java e3d31de
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalAlreadyExistsException.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalDoesNotExistException.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java 90858fb
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 34de6a8
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java eefaf04
>
> Diff: https://reviews.apache.org/r/46363/diff/
>
>
> Testing
> -------
>
> Manually tested in large cluster against MIT KDC and small cluster against Active Directory
>
> # Local test results:
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:04:20.973s
> [INFO] Finished at: Mon Apr 18 21:36:25 EDT 2016
> [INFO] Final Memory: 71M/1554M
> [INFO] ------------------------------------------------------------------------
>
> # Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 46363: Kerberos wizard stuck trying to schedule
service check operation
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46363/
-----------------------------------------------------------
(Updated April 18, 2016, 11:12 p.m.)
Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Sumit Mohanty, Srimanth Gunturi, and Sid Wagle.
Bugs: AMBARI-15961
https://issues.apache.org/jira/browse/AMBARI-15961
Repository: ambari
Description (updated)
-------
Attached jstack after firing up the curl call.
```
curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST -d'{"RequestInfo":{"context":"Kerberos Service Check","command":"KERBEROS_SERVICE_CHECK","operation_level":{"level":"CLUSTER","cluster_name":"c1"}},"Requests/resource_filters":[{"service_name":"KERBEROS"}]}' http://104.196.89.51:8080/api/v1/clusters/c1/requests
```
Behavior:
- Call timedout on the UI and wizard cannot proceed further.
- Exception in the server log after long wait:
```
12 Apr 2016 23:01:45,231 INFO [qtp-ambari-client-818] AmbariManagementControllerImpl:3376 - Received action execution request, clusterName=c1, request=isCommand :true, action :null, command :KERBEROS_SERVICE_CHECK, inputs :{}, resourceFilters: [RequestResourceFilter{serviceName='KERBEROS', componentName='null', hostNames=[]}], exclusive: false, clusterName :c1
12 Apr 2016 23:01:45,409 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HIVE
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: AMBARI_METRICS
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HDFS
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: MAPREDUCE2
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: OOZIE
12 Apr 2016 23:01:45,410 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: TEZ
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: HBASE
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: ZOOKEEPER
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: KERBEROS
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: YARN
12 Apr 2016 23:01:45,411 INFO [qtp-ambari-client-818] AmbariCustomCommandExecutionHelper:1135 - Iterating service type Instance in getCommandJson:: PIG
12 Apr 2016 23:03:49,984 WARN [qtp-ambari-client-818] MITKerberosOperationHandler:434 - Failed to execute kadmin:
Command: /usr/bin/kadmin -s perf-b-3.c.pramod-thangali.internal -p admin -w ******** -r EXAMPLE.COM -q "get_principal admin"
ExitCode: 1
STDOUT: Authenticating as principal admin with password.
STDERR: kadmin: Client not found in Kerberos database while initializing kadmin interface
```
# Solution
- Optimize preparation logic to create principal for Kerberos Service Check
- Optimize logic to create principals in MIT KDC to reduce the time in the Create Principal stage by about 30%
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java 737b0aa
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/ADKerberosOperationHandler.java a1e1544
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/CreatePrincipalsServerAction.java 55e046e
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/DestroyPrincipalsServerAction.java 93daae8
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosMissingAdminCredentialsException.java ddb2769
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosOperationHandler.java e3d31de
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalAlreadyExistsException.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosPrincipalDoesNotExistException.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java 90858fb
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 34de6a8
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java eefaf04
Diff: https://reviews.apache.org/r/46363/diff/
Testing
-------
Manually tested in large cluster against MIT KDC and small cluster against Active Directory
# Local test results:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:04:20.973s
[INFO] Finished at: Mon Apr 18 21:36:25 EDT 2016
[INFO] Final Memory: 71M/1554M
[INFO] ------------------------------------------------------------------------
# Jenkins test results: PENDING
Thanks,
Robert Levas