You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by GitBox <gi...@apache.org> on 2020/01/31 15:38:34 UTC
[GitHub] [knox] moresandeep opened a new pull request #249: KNOX-2213 -
Service Discovery Support for CM UI, API
moresandeep opened a new pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249
## What changes were proposed in this pull request?
Currently when autodiscovery is turned on for CM all the services that are defined in the descriptor are added to the topology except for the CM UI and CM API. They are special cases given CM url is used for service discovery. This PR populates the topology with CM UI and CM API url when they are added to the descriptor.
## How was this patch tested?
This patch was manually Tested.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] moresandeep merged pull request #249: KNOX-2213 - Service
Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
moresandeep merged pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] moresandeep commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
moresandeep commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r374161836
##########
File path: gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscovery.java
##########
@@ -216,18 +220,38 @@ private static ClouderaManagerCluster discoverCluster(DiscoveryApiClient client,
Set<ServiceModel> serviceModels = new HashSet<>();
ApiServiceList serviceList = getClusterServices(servicesResourceApi, clusterName);
+
+ /*
+ Since Cloudera Manager does not have a service for itself, we will add a skeleton CM
+ service so that we can add CM service to topology when auto-discovery is
+ turned on and CM service is selected in the descriptor
+ */
+ final ApiService cmService = new ApiService();
+ cmService.setName(CM_SERVICE_TYPE.toLowerCase(Locale.ROOT));
+ cmService.setType(CM_SERVICE_TYPE);
+ serviceList.addItemsItem(cmService);
Review comment:
Thanks, updated the PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] moresandeep commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
moresandeep commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r373581215
##########
File path: gateway-discovery-cm/src/test/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscoveryTest.java
##########
@@ -562,6 +565,30 @@ public void testNiFiRegistryDiscoverySSL() {
doTestNiFiRegistryDiscovery(true);
}
+ @Test
+ public void testCMDiscovery() {
Review comment:
I see, sure, I'll update this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] smolnar82 commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
smolnar82 commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r374105793
##########
File path: gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscovery.java
##########
@@ -216,18 +220,38 @@ private static ClouderaManagerCluster discoverCluster(DiscoveryApiClient client,
Set<ServiceModel> serviceModels = new HashSet<>();
ApiServiceList serviceList = getClusterServices(servicesResourceApi, clusterName);
+
+ /*
+ Since Cloudera Manager does not have a service for itself, we will add a skeleton CM
+ service so that we can add CM service to topology when auto-discovery is
+ turned on and CM service is selected in the descriptor
+ */
+ final ApiService cmService = new ApiService();
+ cmService.setName(CM_SERVICE_TYPE.toLowerCase(Locale.ROOT));
+ cmService.setType(CM_SERVICE_TYPE);
+ serviceList.addItemsItem(cmService);
Review comment:
`serviceList` can be `null` -> possible NPE here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] smolnar82 commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
smolnar82 commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r374108129
##########
File path: gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscovery.java
##########
@@ -280,7 +304,17 @@ private static ApiRoleList getRoles(RolesResourceApi rolesResourceApi,
String serviceName) {
ApiRoleList roles = null;
try {
- roles = rolesResourceApi.readRoles(clusterName, serviceName, "", VIEW_SUMMARY);
+ /* Populate roles for CM Service since they are not discoverable */
+ if(CM_SERVICE_TYPE.equalsIgnoreCase(serviceName)) {
+ final ApiRoleList roleList = new ApiRoleList();
Review comment:
There is already an `ApiRoleList` type local variable - `roles` - which should be re-used and initiated here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] smolnar82 commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
smolnar82 commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r373570304
##########
File path: gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscovery.java
##########
@@ -202,6 +204,22 @@ private static Cluster discoverCluster(DiscoveryApiClient client, String cluster
}
}
}
+ /* Since Cloudera Manager does not have a service for itself, we will add a skeleton CM
+ service so that we can add CM service to topology when auto-discovery is
+ turned on and CM service is selected in the descriptor */
+ List<ServiceModelGenerator> smgList = serviceModelGenerators
+ .get(CM_SERVICE_TYPE);
+ if (smgList != null) {
+ for (ServiceModelGenerator serviceModelGenerator : smgList) {
+ if (serviceModelGenerator != null) {
+ serviceModelGenerator.setApiClient(client);
+ ServiceModel serviceModel = serviceModelGenerator
+ .generateService(null, null, null, null);
+ serviceModels.add(serviceModel);
+ }
+ }
+ }
+
Review comment:
I believe it would be easier to read and understand later on if you add the CM service above to `serviceList` and modify `getRoles(rolesResourceApi, clusterName, serviceName)` to fetch the roles using the CM REST API for other services and add CM API/UI role manually. Then you would not have to copy-paste the logic which checks service model generators
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] pzampino commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
pzampino commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r373579044
##########
File path: gateway-discovery-cm/src/test/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscoveryTest.java
##########
@@ -562,6 +565,30 @@ public void testNiFiRegistryDiscoverySSL() {
doTestNiFiRegistryDiscovery(true);
}
+ @Test
+ public void testCMDiscovery() {
Review comment:
I think it would be better to separate these, as has been done with other services (e.g., testCMDiscoveryAPI(), testCMDiscoveryUI())
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] pzampino commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
pzampino commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r373576670
##########
File path: gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscovery.java
##########
@@ -202,6 +204,22 @@ private static Cluster discoverCluster(DiscoveryApiClient client, String cluster
}
}
}
+ /* Since Cloudera Manager does not have a service for itself, we will add a skeleton CM
+ service so that we can add CM service to topology when auto-discovery is
+ turned on and CM service is selected in the descriptor */
+ List<ServiceModelGenerator> smgList = serviceModelGenerators
+ .get(CM_SERVICE_TYPE);
+ if (smgList != null) {
+ for (ServiceModelGenerator serviceModelGenerator : smgList) {
+ if (serviceModelGenerator != null) {
+ serviceModelGenerator.setApiClient(client);
+ ServiceModel serviceModel = serviceModelGenerator
+ .generateService(null, null, null, null);
+ serviceModels.add(serviceModel);
+ }
+ }
+ }
+
Review comment:
+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] moresandeep commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
moresandeep commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r373580969
##########
File path: gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscovery.java
##########
@@ -202,6 +204,22 @@ private static Cluster discoverCluster(DiscoveryApiClient client, String cluster
}
}
}
+ /* Since Cloudera Manager does not have a service for itself, we will add a skeleton CM
+ service so that we can add CM service to topology when auto-discovery is
+ turned on and CM service is selected in the descriptor */
+ List<ServiceModelGenerator> smgList = serviceModelGenerators
+ .get(CM_SERVICE_TYPE);
+ if (smgList != null) {
+ for (ServiceModelGenerator serviceModelGenerator : smgList) {
+ if (serviceModelGenerator != null) {
+ serviceModelGenerator.setApiClient(client);
+ ServiceModel serviceModel = serviceModelGenerator
+ .generateService(null, null, null, null);
+ serviceModels.add(serviceModel);
+ }
+ }
+ }
+
Review comment:
Sure, I'll update the PR with this change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] moresandeep commented on a change in pull request #249:
KNOX-2213 - Service Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
moresandeep commented on a change in pull request #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#discussion_r374161915
##########
File path: gateway-discovery-cm/src/main/java/org/apache/knox/gateway/topology/discovery/cm/ClouderaManagerServiceDiscovery.java
##########
@@ -280,7 +304,17 @@ private static ApiRoleList getRoles(RolesResourceApi rolesResourceApi,
String serviceName) {
ApiRoleList roles = null;
try {
- roles = rolesResourceApi.readRoles(clusterName, serviceName, "", VIEW_SUMMARY);
+ /* Populate roles for CM Service since they are not discoverable */
+ if(CM_SERVICE_TYPE.equalsIgnoreCase(serviceName)) {
+ final ApiRoleList roleList = new ApiRoleList();
Review comment:
Thanks, updated the PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] moresandeep commented on issue #249: KNOX-2213 - Service
Discovery Support for CM UI, API
Posted by GitBox <gi...@apache.org>.
moresandeep commented on issue #249: KNOX-2213 - Service Discovery Support for CM UI, API
URL: https://github.com/apache/knox/pull/249#issuecomment-580901364
@pzampino @smolnar82 I addressed your comments, let me know what you think.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services