You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2015/04/29 00:44:24 UTC
Review Request 33642: Add the ability to obtain details about required
Kerberos identities
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33642/
-----------------------------------------------------------
Review request for Ambari, Emil Anca, John Speidel, Robert Nettleton, and Tom Beerbower.
Bugs: AMBARI-10576
https://issues.apache.org/jira/browse/AMBARI-10576
Repository: ambari
Description
-------
Add the ability to obtain details about required Kerberos identities for the cluster. These details should be obtained using a REST API call formatted as a JSON structure.
Resulting JSON block per Kerberos identity:
```
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
```
The data will be converted into CSV-formatted data similar to the file exported from Ambari 1.7.
**Solution**
The following API calls are to be used to obtain the data:
*GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/**
```
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts?fields=kerberos_identities/*",
"items" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1",
"Hosts" : {
"cluster_name" : "c1",
"host_name" : "host1"
},
"kerberos_identities" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
},
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/smokeuser",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "ambari-qa",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
"principal_local_username" : "ambari-qa",
"principal_name" : "ambari-qa@EXAMPLE.COM",
"principal_type" : "USER"
}
},
...
]
},
...
]
}
```
*GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/*&format=csv*
```
host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed
host1,/spnego,HTTP/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
host1,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
host1,datanode_dn,dn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
host1,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
host1,nodemanager_nm,nm/host1@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
host1,namenode_nn,nn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
host1,zookeeper_zk,zookeeper/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
host2,/spnego,HTTP/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
host2,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
host2,datanode_dn,dn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
host2,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
host2,history_server_jhs,jhs/host2@EXAMPLE.COM,SERVICE,mapred,/etc/security/keytabs/jhs.service.keytab,mapred,r,hadoop,,400,true
host2,nodemanager_nm,nm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
host2,secondary_namenode_nn,nn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
host2,resource_manager_rm,rm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/rm.service.keytab,yarn,r,hadoop,,400,true
host2,app_timeline_server_yarn,yarn/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/yarn.service.keytab,yarn,r,hadoop,,400,true
host2,zookeeper_zk,zookeeper/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
...
```
*GET /api/v1/clusters/c1/kerberos_identities?fields=**
```
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities?fields=*",
"items" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
},
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/smokeuser",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "ambari-qa",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
"principal_local_username" : "ambari-qa",
"principal_name" : "ambari-qa@EXAMPLE.COM",
"principal_type" : "USER"
}
},
...
]
}
'''
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5c84d4c
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/HostKerberosIdentityCsvRenderer.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java 02342a8
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java 6dc9e2d
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinition.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostResourceDefinition.java 380e751
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java 776f1f4
ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d
ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java be40bc4
ambari-server/src/main/java/org/apache/ambari/server/api/services/HostKerberosIdentityService.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java aaf3007
ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessorImpl.java 61afee2
ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/CsvSerializer.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java 7f57f7f
ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java 796d64f
ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNodeImpl.java 1739b88
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d6da1eb
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java bdf94a5
ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java f7eb2d9
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 210227e
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProvider.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java 30bac9e
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java 2e5a27d
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java 79537d4
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java 2c0c90a
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java d33adcd
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java 37bf33c
ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java 3f64d9a
ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinitionTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostResourceDefinitionTest.java b3851eb
ambari-server/src/test/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImplTest.java 0daffce
ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/CsvSerializerTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 47f051d
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/33642/diff/
Testing
-------
Manually tested in test cluster
**Local unit test results:**
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44:39.244s
[INFO] Finished at: Tue Apr 28 18:29:01 EDT 2015
[INFO] Final Memory: 60M/1186M
[INFO] ------------------------------------------------------------------------
**Jenkins unit test results: PENDING**
Thanks,
Robert Levas
Re: Review Request 33642: Add the ability to obtain details about
required Kerberos identities
Posted by Robert Nettleton <rn...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33642/#review81963
-----------------------------------------------------------
Ship it!
Ship It!
- Robert Nettleton
On April 28, 2015, 10:44 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33642/
> -----------------------------------------------------------
>
> (Updated April 28, 2015, 10:44 p.m.)
>
>
> Review request for Ambari, Emil Anca, John Speidel, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-10576
> https://issues.apache.org/jira/browse/AMBARI-10576
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Add the ability to obtain details about required Kerberos identities for the cluster. These details should be obtained using a REST API call formatted as a JSON structure.
>
> Resulting JSON block per Kerberos identity:
> ```
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> ```
>
> The data will be converted into CSV-formatted data similar to the file exported from Ambari 1.7.
>
>
> **Solution**
> The following API calls are to be used to obtain the data:
>
> *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/**
> ```
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts?fields=kerberos_identities/*",
> "items" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1",
> "Hosts" : {
> "cluster_name" : "c1",
> "host_name" : "host1"
> },
> "kerberos_identities" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> },
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/smokeuser",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "ambari-qa",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
> "principal_local_username" : "ambari-qa",
> "principal_name" : "ambari-qa@EXAMPLE.COM",
> "principal_type" : "USER"
> }
> },
> ...
> ]
> },
> ...
> ]
> }
> ```
>
> *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/*&format=csv*
> ```
> host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed
> host1,/spnego,HTTP/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
> host1,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
> host1,datanode_dn,dn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
> host1,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
> host1,nodemanager_nm,nm/host1@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
> host1,namenode_nn,nn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
> host1,zookeeper_zk,zookeeper/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
> host2,/spnego,HTTP/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
> host2,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
> host2,datanode_dn,dn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
> host2,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
> host2,history_server_jhs,jhs/host2@EXAMPLE.COM,SERVICE,mapred,/etc/security/keytabs/jhs.service.keytab,mapred,r,hadoop,,400,true
> host2,nodemanager_nm,nm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
> host2,secondary_namenode_nn,nn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
> host2,resource_manager_rm,rm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/rm.service.keytab,yarn,r,hadoop,,400,true
> host2,app_timeline_server_yarn,yarn/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/yarn.service.keytab,yarn,r,hadoop,,400,true
> host2,zookeeper_zk,zookeeper/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
> ...
> ```
>
> *GET /api/v1/clusters/c1/kerberos_identities?fields=**
> ```
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities?fields=*",
> "items" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> },
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/smokeuser",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "ambari-qa",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
> "principal_local_username" : "ambari-qa",
> "principal_name" : "ambari-qa@EXAMPLE.COM",
> "principal_type" : "USER"
> }
> },
> ...
> ]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5c84d4c
> ambari-server/src/main/java/org/apache/ambari/server/api/query/render/HostKerberosIdentityCsvRenderer.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java 02342a8
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java 6dc9e2d
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinition.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostResourceDefinition.java 380e751
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java 776f1f4
> ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java be40bc4
> ambari-server/src/main/java/org/apache/ambari/server/api/services/HostKerberosIdentityService.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java aaf3007
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessorImpl.java 61afee2
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/CsvSerializer.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java 7f57f7f
> ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java 796d64f
> ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNodeImpl.java 1739b88
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d6da1eb
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java bdf94a5
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java f7eb2d9
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 210227e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProvider.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java 30bac9e
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java 2e5a27d
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java 79537d4
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java 2c0c90a
> ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java d33adcd
> ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java 37bf33c
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java 3f64d9a
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinitionTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostResourceDefinitionTest.java b3851eb
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImplTest.java 0daffce
> ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/CsvSerializerTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 47f051d
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/33642/diff/
>
>
> Testing
> -------
>
> Manually tested in test cluster
>
> **Local unit test results:**
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 44:39.244s
> [INFO] Finished at: Tue Apr 28 18:29:01 EDT 2015
> [INFO] Final Memory: 60M/1186M
> [INFO] ------------------------------------------------------------------------
>
> **Jenkins unit test results: PENDING**
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 33642: Add the ability to obtain details about
required Kerberos identities
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33642/#review81974
-----------------------------------------------------------
Ship it!
Ship It!
- Tom Beerbower
On April 29, 2015, 4 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33642/
> -----------------------------------------------------------
>
> (Updated April 29, 2015, 4 p.m.)
>
>
> Review request for Ambari, Emil Anca, John Speidel, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-10576
> https://issues.apache.org/jira/browse/AMBARI-10576
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Add the ability to obtain details about required Kerberos identities for the cluster. These details should be obtained using a REST API call formatted as a JSON structure.
>
> Resulting JSON block per Kerberos identity:
> ```
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> ```
>
> The data will be converted into CSV-formatted data similar to the file exported from Ambari 1.7.
>
>
> **Solution**
> The following API calls are to be used to obtain the data:
>
> *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/**
> ```
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts?fields=kerberos_identities/*",
> "items" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1",
> "Hosts" : {
> "cluster_name" : "c1",
> "host_name" : "host1"
> },
> "kerberos_identities" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> },
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/smokeuser",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "ambari-qa",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
> "principal_local_username" : "ambari-qa",
> "principal_name" : "ambari-qa@EXAMPLE.COM",
> "principal_type" : "USER"
> }
> },
> ...
> ]
> },
> ...
> ]
> }
> ```
>
> *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/*&format=csv*
> ```
> host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed
> host1,/spnego,HTTP/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
> host1,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
> host1,datanode_dn,dn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
> host1,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
> host1,nodemanager_nm,nm/host1@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
> host1,namenode_nn,nn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
> host1,zookeeper_zk,zookeeper/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
> host2,/spnego,HTTP/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
> host2,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
> host2,datanode_dn,dn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
> host2,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
> host2,history_server_jhs,jhs/host2@EXAMPLE.COM,SERVICE,mapred,/etc/security/keytabs/jhs.service.keytab,mapred,r,hadoop,,400,true
> host2,nodemanager_nm,nm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
> host2,secondary_namenode_nn,nn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
> host2,resource_manager_rm,rm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/rm.service.keytab,yarn,r,hadoop,,400,true
> host2,app_timeline_server_yarn,yarn/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/yarn.service.keytab,yarn,r,hadoop,,400,true
> host2,zookeeper_zk,zookeeper/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
> ...
> ```
>
> *GET /api/v1/clusters/c1/kerberos_identities?fields=**
> ```
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities?fields=*",
> "items" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> },
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/smokeuser",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "ambari-qa",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
> "principal_local_username" : "ambari-qa",
> "principal_name" : "ambari-qa@EXAMPLE.COM",
> "principal_type" : "USER"
> }
> },
> ...
> ]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5c84d4c
> ambari-server/src/main/java/org/apache/ambari/server/api/query/render/HostKerberosIdentityCsvRenderer.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java 02342a8
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java 6dc9e2d
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinition.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostResourceDefinition.java 380e751
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java 776f1f4
> ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java be40bc4
> ambari-server/src/main/java/org/apache/ambari/server/api/services/HostKerberosIdentityService.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java aaf3007
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessorImpl.java 61afee2
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/CsvSerializer.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java 7f57f7f
> ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java 796d64f
> ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNodeImpl.java 1739b88
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d6da1eb
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java bdf94a5
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java f7eb2d9
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 210227e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProvider.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java 30bac9e
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java 2e5a27d
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java 79537d4
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java 2c0c90a
> ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java d33adcd
> ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java 37bf33c
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java 3f64d9a
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinitionTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostResourceDefinitionTest.java b3851eb
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImplTest.java 0daffce
> ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/CsvSerializerTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 47f051d
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/33642/diff/
>
>
> Testing
> -------
>
> Manually tested in test cluster
>
> **Local unit test results:**
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 44:39.244s
> [INFO] Finished at: Tue Apr 28 18:29:01 EDT 2015
> [INFO] Final Memory: 60M/1186M
> [INFO] ------------------------------------------------------------------------
>
> **Jenkins unit test results: PENDING**
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 33642: Add the ability to obtain details about
required Kerberos identities
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33642/
-----------------------------------------------------------
(Updated April 29, 2015, noon)
Review request for Ambari, Emil Anca, John Speidel, Robert Nettleton, and Tom Beerbower.
Changes
-------
Addressed reviewer concerns
Bugs: AMBARI-10576
https://issues.apache.org/jira/browse/AMBARI-10576
Repository: ambari
Description
-------
Add the ability to obtain details about required Kerberos identities for the cluster. These details should be obtained using a REST API call formatted as a JSON structure.
Resulting JSON block per Kerberos identity:
```
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
```
The data will be converted into CSV-formatted data similar to the file exported from Ambari 1.7.
**Solution**
The following API calls are to be used to obtain the data:
*GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/**
```
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts?fields=kerberos_identities/*",
"items" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1",
"Hosts" : {
"cluster_name" : "c1",
"host_name" : "host1"
},
"kerberos_identities" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
},
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/smokeuser",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "ambari-qa",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
"principal_local_username" : "ambari-qa",
"principal_name" : "ambari-qa@EXAMPLE.COM",
"principal_type" : "USER"
}
},
...
]
},
...
]
}
```
*GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/*&format=csv*
```
host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed
host1,/spnego,HTTP/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
host1,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
host1,datanode_dn,dn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
host1,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
host1,nodemanager_nm,nm/host1@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
host1,namenode_nn,nn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
host1,zookeeper_zk,zookeeper/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
host2,/spnego,HTTP/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
host2,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
host2,datanode_dn,dn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
host2,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
host2,history_server_jhs,jhs/host2@EXAMPLE.COM,SERVICE,mapred,/etc/security/keytabs/jhs.service.keytab,mapred,r,hadoop,,400,true
host2,nodemanager_nm,nm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
host2,secondary_namenode_nn,nn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
host2,resource_manager_rm,rm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/rm.service.keytab,yarn,r,hadoop,,400,true
host2,app_timeline_server_yarn,yarn/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/yarn.service.keytab,yarn,r,hadoop,,400,true
host2,zookeeper_zk,zookeeper/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
...
```
*GET /api/v1/clusters/c1/kerberos_identities?fields=**
```
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities?fields=*",
"items" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
},
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/smokeuser",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "ambari-qa",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
"principal_local_username" : "ambari-qa",
"principal_name" : "ambari-qa@EXAMPLE.COM",
"principal_type" : "USER"
}
},
...
]
}
```
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5c84d4c
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/HostKerberosIdentityCsvRenderer.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java 02342a8
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java 6dc9e2d
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinition.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostResourceDefinition.java 380e751
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java 776f1f4
ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d
ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java be40bc4
ambari-server/src/main/java/org/apache/ambari/server/api/services/HostKerberosIdentityService.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java aaf3007
ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessorImpl.java 61afee2
ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/CsvSerializer.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java 7f57f7f
ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java 796d64f
ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNodeImpl.java 1739b88
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d6da1eb
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java bdf94a5
ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java f7eb2d9
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 210227e
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProvider.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java 30bac9e
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java 2e5a27d
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java 79537d4
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java 2c0c90a
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java d33adcd
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java 37bf33c
ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java 3f64d9a
ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinitionTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostResourceDefinitionTest.java b3851eb
ambari-server/src/test/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImplTest.java 0daffce
ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/CsvSerializerTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 47f051d
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/33642/diff/
Testing
-------
Manually tested in test cluster
**Local unit test results:**
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44:39.244s
[INFO] Finished at: Tue Apr 28 18:29:01 EDT 2015
[INFO] Final Memory: 60M/1186M
[INFO] ------------------------------------------------------------------------
**Jenkins unit test results: PENDING**
Thanks,
Robert Levas
Re: Review Request 33642: Add the ability to obtain details about
required Kerberos identities
Posted by Robert Levas <rl...@hortonworks.com>.
> On April 29, 2015, 9:38 a.m., Tom Beerbower wrote:
> > Some minor comments. Looks good! At first I thought that using the encoded principal name for the resource id was a bit strange but it's probably better than using a sequence. The CSV renderer is nice... we may be able to reuse that.
Thanks.. I tried to make the CSV serializer relatively reusable, but the format is not as versitle as JSON. My concern was where to add the header and field ordering data. I settled on the TreeNode rather that the Resouce since it was more _infrastructure_ related than _resource_ specific.
> On April 29, 2015, 9:38 a.m., Tom Beerbower wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java, lines 337-339
> > <https://reviews.apache.org/r/33642/diff/1/?file=944511#file944511line337>
> >
> > This silently uses a default value. Does this exception indicate a bigger problem? Should we at least log a warning? I don't know, maybe there is no way that this could occur in the real world.
In the long-run, you are correct - this could indicate an bigger issue. However this method is currently only access internally and should thus never fail. That said, I will throw an exception in the event this method gets called by other means in the future.
> On April 29, 2015, 9:38 a.m., Tom Beerbower wrote:
> > ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java, lines 27-28
> > <https://reviews.apache.org/r/33642/diff/1/?file=944518#file944518line27>
> >
> > Unused imports.
Removing...
> On April 29, 2015, 9:38 a.m., Tom Beerbower wrote:
> > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java, line 51
> > <https://reviews.apache.org/r/33642/diff/1/?file=944521#file944521line51>
> >
> > Unused import.
Removing...
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33642/#review81960
-----------------------------------------------------------
On April 28, 2015, 6:44 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33642/
> -----------------------------------------------------------
>
> (Updated April 28, 2015, 6:44 p.m.)
>
>
> Review request for Ambari, Emil Anca, John Speidel, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-10576
> https://issues.apache.org/jira/browse/AMBARI-10576
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Add the ability to obtain details about required Kerberos identities for the cluster. These details should be obtained using a REST API call formatted as a JSON structure.
>
> Resulting JSON block per Kerberos identity:
> ```
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> ```
>
> The data will be converted into CSV-formatted data similar to the file exported from Ambari 1.7.
>
>
> **Solution**
> The following API calls are to be used to obtain the data:
>
> *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/**
> ```
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts?fields=kerberos_identities/*",
> "items" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1",
> "Hosts" : {
> "cluster_name" : "c1",
> "host_name" : "host1"
> },
> "kerberos_identities" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> },
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/smokeuser",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "ambari-qa",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
> "principal_local_username" : "ambari-qa",
> "principal_name" : "ambari-qa@EXAMPLE.COM",
> "principal_type" : "USER"
> }
> },
> ...
> ]
> },
> ...
> ]
> }
> ```
>
> *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/*&format=csv*
> ```
> host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed
> host1,/spnego,HTTP/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
> host1,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
> host1,datanode_dn,dn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
> host1,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
> host1,nodemanager_nm,nm/host1@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
> host1,namenode_nn,nn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
> host1,zookeeper_zk,zookeeper/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
> host2,/spnego,HTTP/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
> host2,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
> host2,datanode_dn,dn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
> host2,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
> host2,history_server_jhs,jhs/host2@EXAMPLE.COM,SERVICE,mapred,/etc/security/keytabs/jhs.service.keytab,mapred,r,hadoop,,400,true
> host2,nodemanager_nm,nm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
> host2,secondary_namenode_nn,nn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
> host2,resource_manager_rm,rm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/rm.service.keytab,yarn,r,hadoop,,400,true
> host2,app_timeline_server_yarn,yarn/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/yarn.service.keytab,yarn,r,hadoop,,400,true
> host2,zookeeper_zk,zookeeper/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
> ...
> ```
>
> *GET /api/v1/clusters/c1/kerberos_identities?fields=**
> ```
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities?fields=*",
> "items" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> },
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/smokeuser",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "ambari-qa",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
> "principal_local_username" : "ambari-qa",
> "principal_name" : "ambari-qa@EXAMPLE.COM",
> "principal_type" : "USER"
> }
> },
> ...
> ]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5c84d4c
> ambari-server/src/main/java/org/apache/ambari/server/api/query/render/HostKerberosIdentityCsvRenderer.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java 02342a8
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java 6dc9e2d
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinition.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostResourceDefinition.java 380e751
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java 776f1f4
> ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java be40bc4
> ambari-server/src/main/java/org/apache/ambari/server/api/services/HostKerberosIdentityService.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java aaf3007
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessorImpl.java 61afee2
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/CsvSerializer.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java 7f57f7f
> ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java 796d64f
> ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNodeImpl.java 1739b88
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d6da1eb
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java bdf94a5
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java f7eb2d9
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 210227e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProvider.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java 30bac9e
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java 2e5a27d
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java 79537d4
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java 2c0c90a
> ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java d33adcd
> ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java 37bf33c
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java 3f64d9a
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinitionTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostResourceDefinitionTest.java b3851eb
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImplTest.java 0daffce
> ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/CsvSerializerTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 47f051d
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/33642/diff/
>
>
> Testing
> -------
>
> Manually tested in test cluster
>
> **Local unit test results:**
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 44:39.244s
> [INFO] Finished at: Tue Apr 28 18:29:01 EDT 2015
> [INFO] Final Memory: 60M/1186M
> [INFO] ------------------------------------------------------------------------
>
> **Jenkins unit test results: PENDING**
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 33642: Add the ability to obtain details about
required Kerberos identities
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33642/#review81960
-----------------------------------------------------------
Ship it!
Some minor comments. Looks good! At first I thought that using the encoded principal name for the resource id was a bit strange but it's probably better than using a sequence. The CSV renderer is nice... we may be able to reuse that.
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java
<https://reviews.apache.org/r/33642/#comment132525>
This silently uses a default value. Does this exception indicate a bigger problem? Should we at least log a warning? I don't know, maybe there is no way that this could occur in the real world.
ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java
<https://reviews.apache.org/r/33642/#comment132524>
Unused imports.
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java
<https://reviews.apache.org/r/33642/#comment132523>
Unused import.
- Tom Beerbower
On April 28, 2015, 10:44 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33642/
> -----------------------------------------------------------
>
> (Updated April 28, 2015, 10:44 p.m.)
>
>
> Review request for Ambari, Emil Anca, John Speidel, Robert Nettleton, and Tom Beerbower.
>
>
> Bugs: AMBARI-10576
> https://issues.apache.org/jira/browse/AMBARI-10576
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Add the ability to obtain details about required Kerberos identities for the cluster. These details should be obtained using a REST API call formatted as a JSON structure.
>
> Resulting JSON block per Kerberos identity:
> ```
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> ```
>
> The data will be converted into CSV-formatted data similar to the file exported from Ambari 1.7.
>
>
> **Solution**
> The following API calls are to be used to obtain the data:
>
> *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/**
> ```
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts?fields=kerberos_identities/*",
> "items" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1",
> "Hosts" : {
> "cluster_name" : "c1",
> "host_name" : "host1"
> },
> "kerberos_identities" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> },
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/smokeuser",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "ambari-qa",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
> "principal_local_username" : "ambari-qa",
> "principal_name" : "ambari-qa@EXAMPLE.COM",
> "principal_type" : "USER"
> }
> },
> ...
> ]
> },
> ...
> ]
> }
> ```
>
> *GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/*&format=csv*
> ```
> host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed
> host1,/spnego,HTTP/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
> host1,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
> host1,datanode_dn,dn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
> host1,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
> host1,nodemanager_nm,nm/host1@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
> host1,namenode_nn,nn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
> host1,zookeeper_zk,zookeeper/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
> host2,/spnego,HTTP/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
> host2,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
> host2,datanode_dn,dn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
> host2,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
> host2,history_server_jhs,jhs/host2@EXAMPLE.COM,SERVICE,mapred,/etc/security/keytabs/jhs.service.keytab,mapred,r,hadoop,,400,true
> host2,nodemanager_nm,nm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
> host2,secondary_namenode_nn,nn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
> host2,resource_manager_rm,rm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/rm.service.keytab,yarn,r,hadoop,,400,true
> host2,app_timeline_server_yarn,yarn/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/yarn.service.keytab,yarn,r,hadoop,,400,true
> host2,zookeeper_zk,zookeeper/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
> ...
> ```
>
> *GET /api/v1/clusters/c1/kerberos_identities?fields=**
> ```
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities?fields=*",
> "items" : [
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/spnego",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "root",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
> "principal_local_username" : null,
> "principal_name" : "HTTP/host1@EXAMPLE.COM",
> "principal_type" : "SERVICE"
> }
> },
> {
> "href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
> "KerberosIdentity" : {
> "cluster_name" : "c1",
> "description" : "/smokeuser",
> "host_name" : "host1",
> "keytab_file_group" : "hadoop",
> "keytab_file_group_access" : "r",
> "keytab_file_installed" : "true",
> "keytab_file_mode" : "440",
> "keytab_file_owner" : "ambari-qa",
> "keytab_file_owner_access" : "r",
> "keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
> "principal_local_username" : "ambari-qa",
> "principal_name" : "ambari-qa@EXAMPLE.COM",
> "principal_type" : "USER"
> }
> },
> ...
> ]
> }
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5c84d4c
> ambari-server/src/main/java/org/apache/ambari/server/api/query/render/HostKerberosIdentityCsvRenderer.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java 02342a8
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java 6dc9e2d
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinition.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostResourceDefinition.java 380e751
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java 776f1f4
> ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java be40bc4
> ambari-server/src/main/java/org/apache/ambari/server/api/services/HostKerberosIdentityService.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java aaf3007
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessorImpl.java 61afee2
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/CsvSerializer.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java 7f57f7f
> ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java 796d64f
> ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNodeImpl.java 1739b88
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d6da1eb
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java bdf94a5
> ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java f7eb2d9
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 210227e
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProvider.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java 30bac9e
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java 2e5a27d
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java 79537d4
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java 2c0c90a
> ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java d33adcd
> ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java 37bf33c
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java 3f64d9a
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinitionTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostResourceDefinitionTest.java b3851eb
> ambari-server/src/test/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImplTest.java 0daffce
> ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/CsvSerializerTest.java PRE-CREATION
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 47f051d
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/33642/diff/
>
>
> Testing
> -------
>
> Manually tested in test cluster
>
> **Local unit test results:**
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 44:39.244s
> [INFO] Finished at: Tue Apr 28 18:29:01 EDT 2015
> [INFO] Final Memory: 60M/1186M
> [INFO] ------------------------------------------------------------------------
>
> **Jenkins unit test results: PENDING**
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 33642: Add the ability to obtain details about
required Kerberos identities
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33642/
-----------------------------------------------------------
(Updated April 28, 2015, 6:44 p.m.)
Review request for Ambari, Emil Anca, John Speidel, Robert Nettleton, and Tom Beerbower.
Bugs: AMBARI-10576
https://issues.apache.org/jira/browse/AMBARI-10576
Repository: ambari
Description (updated)
-------
Add the ability to obtain details about required Kerberos identities for the cluster. These details should be obtained using a REST API call formatted as a JSON structure.
Resulting JSON block per Kerberos identity:
```
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
```
The data will be converted into CSV-formatted data similar to the file exported from Ambari 1.7.
**Solution**
The following API calls are to be used to obtain the data:
*GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/**
```
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts?fields=kerberos_identities/*",
"items" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1",
"Hosts" : {
"cluster_name" : "c1",
"host_name" : "host1"
},
"kerberos_identities" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
},
{
"href" : "http://ambari:8080/api/v1/clusters/c1/hosts/host1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/smokeuser",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "ambari-qa",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
"principal_local_username" : "ambari-qa",
"principal_name" : "ambari-qa@EXAMPLE.COM",
"principal_type" : "USER"
}
},
...
]
},
...
]
}
```
*GET /api/v1/clusters/c1/hosts?fields=kerberos_identities/*&format=csv*
```
host,description,principal name,principal type,local username,keytab file path,keytab file owner,keytab file owner access,keytab file group,keytab file group access,keytab file mode,keytab file installed
host1,/spnego,HTTP/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
host1,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
host1,datanode_dn,dn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
host1,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
host1,nodemanager_nm,nm/host1@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
host1,namenode_nn,nn/host1@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
host1,zookeeper_zk,zookeeper/host1@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
host2,/spnego,HTTP/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/spnego.service.keytab,root,r,hadoop,r,440,true
host2,/smokeuser,ambari-qa@EXAMPLE.COM,USER,ambari-qa,/etc/security/keytabs/smokeuser.headless.keytab,ambari-qa,r,hadoop,r,440,true
host2,datanode_dn,dn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/dn.service.keytab,hdfs,r,hadoop,,400,true
host2,/hdfs,hdfs@EXAMPLE.COM,USER,hdfs,/etc/security/keytabs/hdfs.headless.keytab,hdfs,r,hadoop,r,440,true
host2,history_server_jhs,jhs/host2@EXAMPLE.COM,SERVICE,mapred,/etc/security/keytabs/jhs.service.keytab,mapred,r,hadoop,,400,true
host2,nodemanager_nm,nm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/nm.service.keytab,yarn,r,hadoop,,400,true
host2,secondary_namenode_nn,nn/host2@EXAMPLE.COM,SERVICE,hdfs,/etc/security/keytabs/nn.service.keytab,hdfs,r,hadoop,,400,true
host2,resource_manager_rm,rm/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/rm.service.keytab,yarn,r,hadoop,,400,true
host2,app_timeline_server_yarn,yarn/host2@EXAMPLE.COM,SERVICE,yarn,/etc/security/keytabs/yarn.service.keytab,yarn,r,hadoop,,400,true
host2,zookeeper_zk,zookeeper/host2@EXAMPLE.COM,SERVICE,,/etc/security/keytabs/zk.service.keytab,zookeeper,r,hadoop,,400,true
...
```
*GET /api/v1/clusters/c1/kerberos_identities?fields=**
```
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities?fields=*",
"items" : [
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/HTTP%2Fhost1%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/spnego",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "root",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/spnego.service.keytab",
"principal_local_username" : null,
"principal_name" : "HTTP/host1@EXAMPLE.COM",
"principal_type" : "SERVICE"
}
},
{
"href" : "http://ambari:8080/api/v1/clusters/c1/kerberos_identities/ambari-qa%40EXAMPLE.COM",
"KerberosIdentity" : {
"cluster_name" : "c1",
"description" : "/smokeuser",
"host_name" : "host1",
"keytab_file_group" : "hadoop",
"keytab_file_group_access" : "r",
"keytab_file_installed" : "true",
"keytab_file_mode" : "440",
"keytab_file_owner" : "ambari-qa",
"keytab_file_owner_access" : "r",
"keytab_file_path" : "/etc/security/keytabs/smokeuser.headless.keytab",
"principal_local_username" : "ambari-qa",
"principal_name" : "ambari-qa@EXAMPLE.COM",
"principal_type" : "USER"
}
},
...
]
}
```
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 5c84d4c
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/HostKerberosIdentityCsvRenderer.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/resources/BaseResourceDefinition.java 02342a8
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostComponentResourceDefinition.java 6dc9e2d
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinition.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/resources/HostResourceDefinition.java 380e751
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java 776f1f4
ambari-server/src/main/java/org/apache/ambari/server/api/services/BaseService.java 3afc23d
ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java be40bc4
ambari-server/src/main/java/org/apache/ambari/server/api/services/HostKerberosIdentityService.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/services/HostService.java aaf3007
ambari-server/src/main/java/org/apache/ambari/server/api/services/ResultPostProcessorImpl.java 61afee2
ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/CsvSerializer.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/api/services/serializers/JsonSerializer.java 7f57f7f
ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNode.java 796d64f
ambari-server/src/main/java/org/apache/ambari/server/api/util/TreeNodeImpl.java 1739b88
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java d6da1eb
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java bdf94a5
ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java f7eb2d9
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 210227e
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProvider.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java 30bac9e
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java 2e5a27d
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosKeytabDescriptor.java 79537d4
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosPrincipalDescriptor.java 2c0c90a
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java d33adcd
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/MinimalRendererTest.java 37bf33c
ambari-server/src/test/java/org/apache/ambari/server/api/resources/BaseResourceDefinitionTest.java 3f64d9a
ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostKerberosIdentityResourceDefinitionTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/api/resources/HostResourceDefinitionTest.java b3851eb
ambari-server/src/test/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImplTest.java 0daffce
ambari-server/src/test/java/org/apache/ambari/server/api/services/HostKerberosIdentityServiceTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/api/services/serializers/CsvSerializerTest.java PRE-CREATION
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java 47f051d
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostKerberosIdentityResourceProviderTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/33642/diff/
Testing
-------
Manually tested in test cluster
**Local unit test results:**
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 44:39.244s
[INFO] Finished at: Tue Apr 28 18:29:01 EDT 2015
[INFO] Final Memory: 60M/1186M
[INFO] ------------------------------------------------------------------------
**Jenkins unit test results: PENDING**
Thanks,
Robert Levas