You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@knox.apache.org by GitBox <gi...@apache.org> on 2020/03/10 15:20:31 UTC
[GitHub] [knox] smolnar82 opened a new pull request #285: KNOX-2284 -
Handling CM descriptors after Knox shared-provider/descriptor/topology
monitors are started
smolnar82 opened a new pull request #285: KNOX-2284 - Handling CM descriptors after Knox shared-provider/descriptor/topology monitors are started
URL: https://github.com/apache/knox/pull/285
## What changes were proposed in this pull request?
Handling CM descriptors after Knox shared-provider/descriptor/topology monitors are started
## How was this patch tested?
Tested manually:
1. redeployed Knox with my changes in place; brand new installation, only the OOTB topologies existed
2. produced a CM descriptor (cm-topologies.cm) with the following content:
```
<configuration>
<property>
<name>topology1</name>
<value>
providerConfigRef=default-providers;
app:knoxauth:param1.name=param1.value;
HIVE:url=http://localhost:456;
HIVE:version=1.0;
HIVE:httpclient.connectionTimeout=5m;
HIVE:httpclient.socketTimeout=100m
</value>
</property>
<property>
<name>topology2</name>
<value>
providerConfigRef=default-providers;
HDFS:url=http://localhost:456;
HDFS:httpclient.connectionTimeout=5m;
HDFS:httpclient.socketTimeout=100m
</value>
</property>
</configuration>
```
3. started Knox and confirmed (from logs and using the Admin UI):
- `$KNOX_HOME/conf/descriptors/topology1.json` and `$KNOX_HOME/conf/descriptors/topology2.json` got persisted
- Knox picked them up and generated `$KNOX_HOME/data/deployments/topology1.topo.170c48f42c0` and `$KNOX_HOME/data/deployments/topology2.topo.170c48f42c0`
Relevant logs:
```
2020-03-10 14:07:50,348 INFO knox.gateway (DefaultTopologyService.java:init(648)) - Monitoring simple descriptors in directory: /Users/smolnar/test/knoxGateway/conf/descriptors
2020-03-10 14:07:50,577 INFO knox.gateway (DefaultTopologyService.java:init(653)) - Monitoring shared provider configurations in directory: /Users/smolnar/test/knoxGateway/conf/shared-providers
...
2020-03-10 14:08:02,899 INFO knox.gateway (GatewayServer.java:start(700)) - Monitoring topologies in directory: /Users/smolnar/test/knoxGateway/conf/topologies
2020-03-10 14:08:02,920 INFO discovery.advanced (AdvancedServiceDiscoveryConfigurationMonitor.java:setupMonitor(73)) - Monitoring /Users/smolnar/test/knoxGateway/conf/auto-discovery-advanced-configuration-* for advanced service discovery configuration changes.
2020-03-10 14:08:02,925 INFO knox.gateway (ClouderaManagerDescriptorMonitor.java:setupMonitor(66)) - Monitoring Cloudera Manager descriptors in /Users/smolnar/test/knoxGateway/conf/descriptors ...
2020-03-10 14:08:02,925 INFO knox.gateway (ClouderaManagerDescriptorParser.java:parse(79)) - Parsing Cloudera Manager descriptor /Users/smolnar/test/knoxGateway/conf/descriptors/cm-topologies.cm. Looking up all topologies...
2020-03-10 14:08:02,925 INFO knox.gateway (GatewayServer.java:startGateway(392)) - Started gateway on port 8,443.
2020-03-10 14:08:02,936 INFO knox.gateway (ClouderaManagerDescriptorParser.java:parse(84)) - Found Knox descriptors topology1, topology2 in /Users/smolnar/test/knoxGateway/conf/descriptors/cm-topologies.cm
2020-03-10 14:08:03,125 INFO knox.gateway (ClouderaManagerDescriptorMonitor.java:lambda$1(100)) - Saved Knox descriptor /Users/smolnar/test/knoxGateway/conf/descriptors/topology1.json
2020-03-10 14:08:03,125 INFO knox.gateway (ClouderaManagerDescriptorMonitor.java:lambda$1(100)) - Saved Knox descriptor /Users/smolnar/test/knoxGateway/conf/descriptors/topology2.json
2020-03-10 14:08:08,011 INFO knox.gateway (DefaultTopologyService.java:onFileChange(872)) - Generated topology topology1.xml because the associated descriptor topology1.json changed.
...
2020-03-10 14:08:14,278 INFO knox.gateway (GatewayServer.java:handleCreateDeployment(1013)) - Deploying topology topology1 to /Users/smolnar/test/knoxGateway/data/deployments/topology1.topo.170c48f42c0
2020-03-10 14:08:14,278 INFO knox.gateway (GatewayServer.java:internalDeactivateTopology(932)) - Deactivating topology topology1
2020-03-10 14:08:14,450 INFO knox.gateway (DefaultGatewayServices.java:initializeContribution(161)) - Creating credential store for the cluster: topology1
2020-03-10 14:08:14,753 INFO knox.gateway (GatewayServer.java:internalActivateTopology(898)) - Activating topology topology1
2020-03-10 14:08:14,753 INFO knox.gateway (GatewayServer.java:internalActivateArchive(908)) - Activating topology topology1 archive %2Fknoxauth
2020-03-10 14:08:14,833 INFO knox.gateway (GatewayServer.java:handleCreateDeployment(1013)) - Deploying topology topology2 to /Users/smolnar/test/knoxGateway/data/deployments/topology2.topo.170c48f42c0
2020-03-10 14:08:14,833 INFO knox.gateway (GatewayServer.java:internalDeactivateTopology(932)) - Deactivating topology topology2
2020-03-10 14:08:14,940 INFO knox.gateway (GatewayServer.java:internalActivateTopology(898)) - Activating topology topology2
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [knox] smolnar82 merged pull request #285: KNOX-2284 - Handling CM
descriptors after Knox shared-provider/descriptor/topology monitors are
started
Posted by GitBox <gi...@apache.org>.
smolnar82 merged pull request #285: KNOX-2284 - Handling CM descriptors after Knox shared-provider/descriptor/topology monitors are started
URL: https://github.com/apache/knox/pull/285
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services