You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/01/16 11:03:00 UTC
[jira] [Work logged] (KNOX-2186) Cloudera Manager integration -
filter services
[ https://issues.apache.org/jira/browse/KNOX-2186?focusedWorklogId=372939&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372939 ]
ASF GitHub Bot logged work on KNOX-2186:
----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Jan/20 11:02
Start Date: 16/Jan/20 11:02
Worklog Time Spent: 10m
Work Description: smolnar82 commented on pull request #238: KNOX-2186 - Advanced service discovery configuration handling
URL: https://github.com/apache/knox/pull/238
## What changes were proposed in this pull request?
Implemented CM specific advanced service discovery feature as described in [KNOX-2186](https://issues.apache.org/jira/browse/KNOX-2186).
## How was this patch tested?
Updated and ran JUnit tests:
```
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18:37 min (Wall Clock)
[INFO] Finished at: 2020-01-16T11:47:14+01:00
[INFO] Final Memory: 421M/2310M
[INFO] ------------------------------------------------------------------------
```
Additionally, I tested new features manually. I've used the following two files:
`/Users/smolnar/test/knoxGateway/conf/descriptors/testDescriptor.cm`
```
<configuration>
<property>
<name>topology1</name>
<value>
discoveryType=ClouderaManager;
discoveryAddress=http://host:123;
discoveryUser=user;
discoveryPasswordAlias=alias;
cluster=Cluster 1;
providerConfigRef=topology1-provider;
app:knoxauth:param1.name=param1.value;
HIVE:url=http://localhost:456;
HIVE:version=1.0;
HIVE:httpclient.connectionTimeout=5m;
HIVE:httpclient.socketTimeout=100m;
MY_CUSTOM_SERVICE:url=https://custom_host:custom_port
</value>
</property>
<property>
<name>topology2</name>
<value>
providerConfigRef=topology2-provider;
ATLAS_API:url=http://localhost:456;
ATLAS_API:httpclient.connectionTimeout=15m;
ATLAS_API:httpclient.socketTimeout=100m;
RANGERUI:url=http://localhost:789;
RANGER:url=http://localhost:rangerport
</value>
</property>
</configuration>
```
`/Users/smolnar/test/knoxGateway/conf/auto-discovery-advanced-configuration.properties`
```
gateway.auto.discovery.enforce.required.services=true
gateway.auto.discovery.enabled.ATLAS_API=true
gateway.auto.discovery.enabled.HIVE=false
gateway.auto.discovery.enabled.RANGERUI=true
gateway.auto.discovery.enabled.RANGER=false
```
During my test rounds, I modified the content of both files, combining them to cover all possible use-cases (e.g. enforcing required services, disabled enforcement, disabled/enabled services, used custom services, etc...). In all cases, I confirmed that CM specific monitors triggered the necessary actions and the expected descriptors were generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 372939)
Remaining Estimate: 0h
Time Spent: 10m
> Cloudera Manager integration - filter services
> ----------------------------------------------
>
> Key: KNOX-2186
> URL: https://issues.apache.org/jira/browse/KNOX-2186
> Project: Apache Knox
> Issue Type: New Feature
> Components: Server
> Affects Versions: 1.4.0
> Reporter: Sandor Molnar
> Assignee: Sandor Molnar
> Priority: Major
> Fix For: 1.4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With KNOX-2160, Knos is capable of processing a Hadoop XML configuration file and turn its content into Knox providers.
> It'd be beneficial to add support for the following use cases that are also Cloudera Manager integration specific:
> *1.)* Cloudera Manager reports if auto-discovery is {{enabled}} for each known services. That is, a list of boolean properties can be generated by CM indicating if {{SERVICE_X}} is enabled or not in the following form: {{gateway.auto.discovery.enabled.SERVICE_NAME=[true|false]}}
> The new Hadoop XML configuration parser should take this information into account, and add a certain service into the generated Knox descriptor only if that service is explicitly {{enabled}} or there is no boolean flag within the CM generated properties with that service name (indicating an unknown - custom - service)
> *2.)* There are some services - mainly UI services - that are not working without some more required services in place (mainly their API counterpart). For instance: {{RANGERUI}} won't work properly if {{RANGER}} is not available.
> The new Hadoop XML configuration parser should be modified to not to add any service into the generated Knox descriptor unless
> * all required services are available (if any)
> * all required services are enabled (see the previous point)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)