You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by YolandaMDavis <gi...@git.apache.org> on 2017/04/12 19:28:19 UTC
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
GitHub user YolandaMDavis opened a pull request:
https://github.com/apache/nifi/pull/1669
NIFI-3695 - created the nifi admin toolkit which includes shell scrip\u2026
\u2026ts and classes to support notification and basic node management in standalone and clustered nifi.
Documentation
[Admin Toolkit for NiFi.docx](https://github.com/apache/nifi/files/917612/Admin.Toolkit.for.NiFi.docx)
and use case information is attached
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [X] Is your initial contribution a single, squashed commit?
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [X] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/YolandaMDavis/nifi NIFI-3695
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1669.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1669
----
commit c6e66279d0d7e9b803f51ce9715262b51e25d8d1
Author: Yolanda M. Davis <yo...@gmail.com>
Date: 2017-02-07T15:28:15Z
NIFI-3695 - created the nifi admin toolkit which includes shell scripts and classes to support notification and basic node management in standalone and clustered nifi.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on the issue:
https://github.com/apache/nifi/pull/1669
For the keystore, truststores in test resources, is it possible we could use the tls-toolkit to generate those on demand?
https://github.com/apache/nifi-minifi/blob/master/minifi-c2/minifi-c2-integration-tests/src/test/java/org/apache/nifi/minifi/c2/integration/test/AbstractTestSecure.java#L71
That should keep them from expiring or needing to be changed later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by YolandaMDavis <gi...@git.apache.org>.
Github user YolandaMDavis commented on the issue:
https://github.com/apache/nifi/pull/1669
@brosander thanks so much for reviewing! Concerning documentation I do have a link attached. Will also look at the code changes you suggested.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r111806902
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/groovy/org/apache/nifi/web/StandardNiFiServiceFacadeSpec.groovy ---
@@ -49,6 +53,7 @@ class StandardNiFiServiceFacadeSpec extends Specification {
SecurityContextHolder.getContext().setAuthentication(null);
}
--- End diff --
Is there an issue with these ignored tests?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on the issue:
https://github.com/apache/nifi/pull/1669
+1 looks good to me, seems to be working, @alopresto did @YolandaMDavis resolve your concerns?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r111806379
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/manager/BulletinMerger.java ---
@@ -71,7 +74,12 @@ public int compare(BulletinEntity o1, BulletinEntity o2) {
}
}
- Collections.sort(bulletinEntities, (BulletinEntity o1, BulletinEntity o2) -> {
+ final List<BulletinEntity> entities = Lists.newArrayList();
+
+ final Map<String,List<BulletinEntity>> groupingEntities = bulletinEntities.stream().collect(Collectors.groupingBy(b -> b.getBulletin().getMessage()));
+ groupingEntities.keySet().stream().forEach(m -> entities.add(groupingEntities.get(m).get(0)));
--- End diff --
Would streaming values make this operation a little easier?
Something like:
```
groupingEntities.values().stream().map(e -> e.get(0)).forEach(entities::add);
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r111807253
--- Diff: nifi-toolkit/nifi-toolkit-admin/pom.xml ---
@@ -0,0 +1,189 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor
+license agreements. See the NOTICE file distributed with this work for additional
+information regarding copyright ownership. The ASF licenses this file to
+You under the Apache License, Version 2.0 (the "License"); you may not use
+this file except in compliance with the License. You may obtain a copy of
+the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required
+by applicable law or agreed to in writing, software distributed under the
+License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
+OF ANY KIND, either express or implied. See the License for the specific
+language governing permissions and limitations under the License. -->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-toolkit</artifactId>
+ <version>1.2.0-SNAPSHOT</version>
+ </parent>
+
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>nifi-toolkit-admin</artifactId>
+
+ <dependencies>
+ <dependency>
+ <groupId>commons-cli</groupId>
+ <artifactId>commons-cli</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-toolkit-tls</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.sun.jersey</groupId>
+ <artifactId>jersey-client</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-client-dto</artifactId>
+ <version>1.2.0-SNAPSHOT</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-properties</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-properties-loader</artifactId>
+ <exclusions>
+ <exclusion>
+ <groupId>ch.qos.logback</groupId>
+ <artifactId>logback-classic</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
+ <dependency>
+ <groupId>org.codehaus.jackson</groupId>
+ <artifactId>jackson-mapper-asl</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.jaxrs</groupId>
+ <artifactId>jackson-jaxrs-json-provider</artifactId>
+ <version>RELEASE</version>
+ </dependency>
+ <dependency>
+ <groupId>com.sun.jersey</groupId>
+ <artifactId>jersey-bundle</artifactId>
+ <version>RELEASE</version>
+ </dependency>
+ <dependency>
+ <groupId>com.sun.jersey</groupId>
+ <artifactId>jersey-json</artifactId>
+ <version>RELEASE</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.commons</groupId>
+ <artifactId>commons-compress</artifactId>
+ </dependency>
+ <!-- Spock testing dependencies-->
+ <dependency>
+ <groupId>com.github.stefanbirkner</groupId>
+ <artifactId>system-rules</artifactId>
+ <version>1.16.0</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.spockframework</groupId>
+ <artifactId>spock-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>cglib</groupId>
+ <artifactId>cglib-nodep</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <executions>
+ <execution>
+ <goals>
+ <goal>compile</goal>
+ <goal>testCompile</goal>
+ </goals>
+ <configuration>
+ <compilerId>groovy-eclipse-compiler</compilerId>
+ </configuration>
+ </execution>
+ </executions>
+ <configuration>
+ <source>1.8</source>
+ <target>1.8</target>
+ </configuration>
+ <dependencies>
+ <dependency>
+ <groupId>org.codehaus.groovy</groupId>
+ <artifactId>groovy-eclipse-compiler</artifactId>
+ <version>2.9.2-01</version>
+ </dependency>
+ <dependency>
+ <groupId>org.codehaus.groovy</groupId>
+ <artifactId>groovy-eclipse-batch</artifactId>
+ <version>2.4.3-01</version>
+ </dependency>
+ </dependencies>
+ </plugin>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>build-helper-maven-plugin</artifactId>
+ <version>1.5</version>
+ <executions>
+ <execution>
+ <id>add-source</id>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>add-source</goal>
+ </goals>
+ <configuration>
+ <sources>
+ <source>src/main/groovy</source>
+ </sources>
+ </configuration>
+ </execution>
+ <execution>
+ <id>add-test-source</id>
+ <phase>generate-test-sources</phase>
+ <goals>
+ <goal>add-test-source</goal>
+ </goals>
+ <configuration>
+ <sources>
+ <source>src/test/groovy</source>
+ </sources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.rat</groupId>
+ <artifactId>apache-rat-plugin</artifactId>
+ <configuration>
+ <excludes combine.children="append">
+ <exclude>src/test/resources/conf/nifi.properties</exclude>
--- End diff --
Not a huge deal, but the properties and xml files could have the license headers so they wouldn't have to be ignored by rat
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r112568901
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/test/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientFactorySpec.groovy ---
@@ -71,6 +93,71 @@ class NiFiClientFactorySpec extends Specification {
}
+ def "should verify CN in certificate based on subjectDN"(){
--- End diff --
I like these tests. If you add a test for the manipulated DN I posted above, you can verify that the `LdapName` implementation works with that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r112568223
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/main/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientFactory.groovy ---
@@ -116,22 +121,27 @@ class NiFiClientFactory implements ClientFactory{
return sslContext;
}
- private static class NiFiHostnameVerifier implements HostnameVerifier {
+ static class NiFiHostnameVerifier implements HostnameVerifier {
@Override
public boolean verify(final String hostname, final SSLSession ssls) {
try {
- for (final Certificate peerCertificate : ssls.getPeerCertificates()) {
- if (peerCertificate instanceof X509Certificate) {
- final X509Certificate x509Cert = (X509Certificate) peerCertificate;
- final List<String> subjectAltNames = getSubjectAlternativeNames(x509Cert);
- if (subjectAltNames.contains(hostname.toLowerCase())) {
- return true;
- }
+
+ final Certificate peerCertificate = ssls.getPeerCertificates()[0]
+ final X509Certificate x509Cert = CertificateUtils.convertAbstractX509Certificate(peerCertificate)
+ final String dn = x509Cert.getSubjectDN().getName().trim()
+ final String cn = dn.tokenize(",").find { cn -> cn.startsWith("CN=")}
+
+ if(StringUtils.isNoneEmpty(cn) && StringUtils.substringAfter(cn,"=").toLowerCase().equals(hostname.toLowerCase())){
--- End diff --
I think `StringUtils.isNoneEmpty(cn)` should be `StringUtils.isNotEmpty(cn)` -- probably an autocomplete issue. You can also use `equalsIgnoreCase()` rather than having to convert both strings to lowercase.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on the issue:
https://github.com/apache/nifi/pull/1669
Would it be difficult to create the other binary resources in test setup? If not it might be preferable to having zips and tars in source control.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by YolandaMDavis <gi...@git.apache.org>.
Github user YolandaMDavis commented on the issue:
https://github.com/apache/nifi/pull/1669
@brosander I've chatted with @thesolson about documentation and she was willing to help there. She will create a separate request in Jira for that effort to add admin-toolkit info to the admin guide.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by YolandaMDavis <gi...@git.apache.org>.
Github user YolandaMDavis commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r112242565
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/main/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientUtil.groovy ---
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.toolkit.admin.client
+
+import com.google.common.collect.Lists
+import com.sun.jersey.api.client.Client
+import com.sun.jersey.api.client.ClientResponse
+import com.sun.jersey.api.client.WebResource
+import org.apache.nifi.util.NiFiProperties
+import org.apache.nifi.util.StringUtils
+import org.apache.nifi.web.api.dto.NodeDTO
+import org.apache.nifi.web.api.entity.ClusterEntity
+import org.slf4j.Logger
+import org.slf4j.LoggerFactory
+
+public class NiFiClientUtil {
+
+ private static final Logger logger = LoggerFactory.getLogger(NiFiClientUtil.class)
+ private final static String GET_CLUSTER_ENDPOINT ="/nifi-api/controller/cluster"
+
+ public static Boolean isCluster(final NiFiProperties niFiProperties){
+ String clusterNode = niFiProperties.getProperty(NiFiProperties.CLUSTER_IS_NODE)
+ return Boolean.valueOf(clusterNode)
+ }
+
+ public static String getUrl(NiFiProperties niFiProperties, String endpoint){
+
+ final StringBuilder urlBuilder = new StringBuilder();
+
+ if(!StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT))){
+ urlBuilder.append("https://")
+ urlBuilder.append(StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_HOST)) ? "localhost": niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_HOST))
+ urlBuilder.append(":")
+ urlBuilder.append(StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT)) ? "8081" : niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT))
+ }else{
+ urlBuilder.append("http://")
+ urlBuilder.append(StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTP_HOST)) ? "localhost": niFiProperties.getProperty(NiFiProperties.WEB_HTTP_HOST))
+ urlBuilder.append(":")
+ urlBuilder.append(StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT)) ? "8080": niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT))
+ }
+
+ if(!StringUtils.isEmpty(endpoint)) {
+ urlBuilder.append(endpoint)
+ }
+
+ urlBuilder.toString()
+ }
+
+ public static String getUrl(NiFiProperties niFiProperties, NodeDTO nodeDTO, String endpoint){
+
+ final StringBuilder urlBuilder = new StringBuilder();
+ if(!StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT))){
+ urlBuilder.append("https://")
+
+ }else{
+ urlBuilder.append("http://")
+ }
+ urlBuilder.append(nodeDTO.address)
+ urlBuilder.append(":")
+ urlBuilder.append(nodeDTO.apiPort)
+
+ if(!StringUtils.isEmpty(endpoint)) {
+ urlBuilder.append(endpoint)
+ }
+
+ urlBuilder.toString()
+ }
+
+ public static ClusterEntity getCluster(final Client client, NiFiProperties niFiProperties, List<String> activeUrls){
+
+ if(activeUrls.isEmpty()){
+ final String url = getUrl(niFiProperties,null)
+ activeUrls.add(url)
+ }
+
+ for(String activeUrl: activeUrls) {
+
+ try {
+
+ String url = activeUrl + GET_CLUSTER_ENDPOINT
+ final WebResource webResource = client.resource(url)
+ final ClientResponse response = webResource.type("application/json").get(ClientResponse.class)
+
+ Integer status = response.getStatus()
+
+ if (status != 200) {
+ if (status == 404) {
+ logger.warn("This node is not attached to a cluster. Please connect to a node that is attached to the cluster for information")
+ } else {
+ logger.warn("Failed with HTTP error code: {}, message: {}", status, response.getStatusInfo().getReasonPhrase())
+ }
+ } else if (status == 200) {
+ return response.getEntity(ClusterEntity.class)
+ }
+
+ }catch(Exception ex){
+ logger.warn("Exception occurred during connection attempt: {}",ex.localizedMessage)
+ }
+
+ }
+
+ throw new RuntimeException("Unable to obtain cluster information")
+
+ }
+
+ public static List<String> getActiveClusterUrls(final Client client, NiFiProperties niFiProperties){
+
+ final ClusterEntity clusterEntity = getCluster(client, niFiProperties, Lists.newArrayList())
+ final List<NodeDTO> activeNodes = clusterEntity.cluster.nodes.findAll{ it.status == "CONNECTED" }
+ final List<String> activeUrls = Lists.newArrayList()
+
+ activeNodes.each {
+ activeUrls.add(getUrl(niFiProperties,it, null))
+ }
+ activeUrls
+ }
+
+ public static String convertToJson(NodeDTO nodeDTO){
--- End diff --
@brosander I originally had an issue with Jackson and date serialization however I figured out what I needed to do. It's now in use in the recent commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by YolandaMDavis <gi...@git.apache.org>.
Github user YolandaMDavis commented on the issue:
https://github.com/apache/nifi/pull/1669
@brosander concerning the binaries they have a very small footprint so I wasn't too concerned about including them (32K, 27K and 1K). I will work to incorporate the cert generation test and documentation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on the issue:
https://github.com/apache/nifi/pull/1669
Hey @YolandaMDavis this looks like a really good addition to the nifi toolkit! Being able to send bulletins and manage nodes via the command line will likely be a really nice-to-have feature for admins.
Put feedback above. Please let me know if you need more info.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r112566667
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/main/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientFactory.groovy ---
@@ -116,22 +121,27 @@ class NiFiClientFactory implements ClientFactory{
return sslContext;
}
- private static class NiFiHostnameVerifier implements HostnameVerifier {
+ static class NiFiHostnameVerifier implements HostnameVerifier {
@Override
public boolean verify(final String hostname, final SSLSession ssls) {
try {
- for (final Certificate peerCertificate : ssls.getPeerCertificates()) {
- if (peerCertificate instanceof X509Certificate) {
- final X509Certificate x509Cert = (X509Certificate) peerCertificate;
- final List<String> subjectAltNames = getSubjectAlternativeNames(x509Cert);
- if (subjectAltNames.contains(hostname.toLowerCase())) {
- return true;
- }
+
+ final Certificate peerCertificate = ssls.getPeerCertificates()[0]
--- End diff --
I think this could throw an `ArrayIndexOutOfBoundsException` -- we should check the presence/size first and log/throw an exception.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r112304681
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/main/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientFactory.groovy ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.toolkit.admin.client
+
+import com.sun.jersey.api.client.Client
+import com.sun.jersey.api.client.config.ClientConfig
+import com.sun.jersey.api.client.config.DefaultClientConfig
+import com.sun.jersey.client.urlconnection.HTTPSProperties
+import org.apache.commons.lang3.StringUtils
+import org.apache.nifi.util.NiFiProperties
+import javax.net.ssl.HostnameVerifier
+import javax.net.ssl.KeyManagerFactory
+import javax.net.ssl.SSLContext
+import javax.net.ssl.SSLPeerUnverifiedException
+import javax.net.ssl.SSLSession
+import javax.net.ssl.TrustManagerFactory
+import java.security.KeyManagementException
+import java.security.KeyStore
+import java.security.KeyStoreException
+import java.security.NoSuchAlgorithmException
+import java.security.SecureRandom
+import java.security.UnrecoverableKeyException
+import java.security.cert.Certificate
+import java.security.cert.CertificateException
+import java.security.cert.CertificateParsingException
+import java.security.cert.X509Certificate
+
+class NiFiClientFactory implements ClientFactory{
+
+ static enum NiFiAuthType{ NONE, SSL }
+
+ public Client getClient(NiFiProperties niFiProperties, String nifiInstallDir) throws Exception {
+
+ final String authTypeStr = StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_HOST)) && StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT)) ? NiFiAuthType.NONE : NiFiAuthType.SSL;
+ final NiFiAuthType authType = NiFiAuthType.valueOf(authTypeStr);
+
+ SSLContext sslContext = null;
+
+ if (NiFiAuthType.SSL.equals(authType)) {
+ String keystore = niFiProperties.getProperty(NiFiProperties.SECURITY_KEYSTORE);
+ final String keystoreType = niFiProperties.getProperty(NiFiProperties.SECURITY_KEYSTORE_TYPE);
+ final String keystorePassword = niFiProperties.getProperty(NiFiProperties.SECURITY_KEYSTORE_PASSWD);
+ String truststore = niFiProperties.getProperty(NiFiProperties.SECURITY_TRUSTSTORE);
+ final String truststoreType = niFiProperties.getProperty(NiFiProperties.SECURITY_TRUSTSTORE_TYPE);
+ final String truststorePassword = niFiProperties.getProperty(NiFiProperties.SECURITY_TRUSTSTORE_PASSWD);
+
+ if(keystore.startsWith("./")){
+ keystore = keystore.replace("./",nifiInstallDir+"/")
+ }
+ if(truststore.startsWith("./")){
+ truststore = truststore.replace("./",nifiInstallDir+"/")
+ }
+
+ sslContext = createSslContext(
+ keystore.trim(),
+ keystorePassword.trim().toCharArray(),
+ keystoreType.trim(),
+ truststore.trim(),
+ truststorePassword.trim().toCharArray(),
+ truststoreType.trim(),
+ "TLS");
+ }
+
+ final ClientConfig config = new DefaultClientConfig();
+
+ if (sslContext != null) {
+ config.getProperties().put(HTTPSProperties.PROPERTY_HTTPS_PROPERTIES,new HTTPSProperties(new NiFiHostnameVerifier(), sslContext));
+ }
+
+ return Client.create(config);
+
+ }
+
+
+ private static SSLContext createSslContext(
+ final String keystore, final char[] keystorePasswd, final String keystoreType,
+ final String truststore, final char[] truststorePasswd, final String truststoreType,
+ final String protocol)
+ throws KeyStoreException, IOException, NoSuchAlgorithmException, CertificateException,
+ UnrecoverableKeyException, KeyManagementException {
+
+ // prepare the keystore
+ final KeyStore keyStore = KeyStore.getInstance(keystoreType);
+ final InputStream keyStoreStream = new FileInputStream(keystore)
+ keyStore.load(keyStoreStream, keystorePasswd);
+
+
+ final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
+ keyManagerFactory.init(keyStore, keystorePasswd);
+
+ // prepare the truststore
+ final KeyStore trustStore = KeyStore.getInstance(truststoreType);
+ final InputStream trustStoreStream = new FileInputStream(truststore)
+ trustStore.load(trustStoreStream, truststorePasswd);
+
+ final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
+ trustManagerFactory.init(trustStore);
+
+ // initialize the ssl context
+ final SSLContext sslContext = SSLContext.getInstance(protocol);
+ sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), new SecureRandom());
+ return sslContext;
+ }
+
+ private static class NiFiHostnameVerifier implements HostnameVerifier {
+
+ @Override
+ public boolean verify(final String hostname, final SSLSession ssls) {
+ try {
+ for (final Certificate peerCertificate : ssls.getPeerCertificates()) {
+ if (peerCertificate instanceof X509Certificate) {
+ final X509Certificate x509Cert = (X509Certificate) peerCertificate;
+ final List<String> subjectAltNames = getSubjectAlternativeNames(x509Cert);
--- End diff --
The hostname should first be compared to the hostname in the subject DN of the certificate before resorting to the SANs. There are two scenarios where this would return incorrectly -- the cert hostname matches but no SANs match (false negative) or the cert has no SANs (false negative). See [`X509Certificate.getSubjectX500Principal()`](https://docs.oracle.com/javase/7/docs/api/java/security/cert/X509Certificate.html#getSubjectX500Principal()) to access the subject DN directly and [`X500Principal.getName()`](https://docs.oracle.com/javase/7/docs/api/javax/security/auth/x500/X500Principal.html#getName()) to extract the hostname (`CN` value) from the principal for comparison. If this comparison fails, then check the SANs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by YolandaMDavis <gi...@git.apache.org>.
Github user YolandaMDavis commented on the issue:
https://github.com/apache/nifi/pull/1669
@alopresto thanks so much for your feedback. I've made changes for your review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r112303857
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/main/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientFactory.groovy ---
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.toolkit.admin.client
+
+import com.sun.jersey.api.client.Client
+import com.sun.jersey.api.client.config.ClientConfig
+import com.sun.jersey.api.client.config.DefaultClientConfig
+import com.sun.jersey.client.urlconnection.HTTPSProperties
+import org.apache.commons.lang3.StringUtils
+import org.apache.nifi.util.NiFiProperties
+import javax.net.ssl.HostnameVerifier
+import javax.net.ssl.KeyManagerFactory
+import javax.net.ssl.SSLContext
+import javax.net.ssl.SSLPeerUnverifiedException
+import javax.net.ssl.SSLSession
+import javax.net.ssl.TrustManagerFactory
+import java.security.KeyManagementException
+import java.security.KeyStore
+import java.security.KeyStoreException
+import java.security.NoSuchAlgorithmException
+import java.security.SecureRandom
+import java.security.UnrecoverableKeyException
+import java.security.cert.Certificate
+import java.security.cert.CertificateException
+import java.security.cert.CertificateParsingException
+import java.security.cert.X509Certificate
+
+class NiFiClientFactory implements ClientFactory{
+
+ static enum NiFiAuthType{ NONE, SSL }
+
+ public Client getClient(NiFiProperties niFiProperties, String nifiInstallDir) throws Exception {
+
+ final String authTypeStr = StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_HOST)) && StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT)) ? NiFiAuthType.NONE : NiFiAuthType.SSL;
+ final NiFiAuthType authType = NiFiAuthType.valueOf(authTypeStr);
+
+ SSLContext sslContext = null;
+
+ if (NiFiAuthType.SSL.equals(authType)) {
+ String keystore = niFiProperties.getProperty(NiFiProperties.SECURITY_KEYSTORE);
+ final String keystoreType = niFiProperties.getProperty(NiFiProperties.SECURITY_KEYSTORE_TYPE);
+ final String keystorePassword = niFiProperties.getProperty(NiFiProperties.SECURITY_KEYSTORE_PASSWD);
+ String truststore = niFiProperties.getProperty(NiFiProperties.SECURITY_TRUSTSTORE);
+ final String truststoreType = niFiProperties.getProperty(NiFiProperties.SECURITY_TRUSTSTORE_TYPE);
+ final String truststorePassword = niFiProperties.getProperty(NiFiProperties.SECURITY_TRUSTSTORE_PASSWD);
+
+ if(keystore.startsWith("./")){
+ keystore = keystore.replace("./",nifiInstallDir+"/")
+ }
+ if(truststore.startsWith("./")){
+ truststore = truststore.replace("./",nifiInstallDir+"/")
+ }
+
+ sslContext = createSslContext(
+ keystore.trim(),
+ keystorePassword.trim().toCharArray(),
+ keystoreType.trim(),
+ truststore.trim(),
+ truststorePassword.trim().toCharArray(),
+ truststoreType.trim(),
+ "TLS");
+ }
+
+ final ClientConfig config = new DefaultClientConfig();
+
+ if (sslContext != null) {
+ config.getProperties().put(HTTPSProperties.PROPERTY_HTTPS_PROPERTIES,new HTTPSProperties(new NiFiHostnameVerifier(), sslContext));
+ }
+
+ return Client.create(config);
+
+ }
+
+
+ private static SSLContext createSslContext(
+ final String keystore, final char[] keystorePasswd, final String keystoreType,
+ final String truststore, final char[] truststorePasswd, final String truststoreType,
+ final String protocol)
+ throws KeyStoreException, IOException, NoSuchAlgorithmException, CertificateException,
+ UnrecoverableKeyException, KeyManagementException {
+
+ // prepare the keystore
+ final KeyStore keyStore = KeyStore.getInstance(keystoreType);
+ final InputStream keyStoreStream = new FileInputStream(keystore)
+ keyStore.load(keyStoreStream, keystorePasswd);
+
+
+ final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
+ keyManagerFactory.init(keyStore, keystorePasswd);
+
+ // prepare the truststore
+ final KeyStore trustStore = KeyStore.getInstance(truststoreType);
+ final InputStream trustStoreStream = new FileInputStream(truststore)
+ trustStore.load(trustStoreStream, truststorePasswd);
+
+ final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
+ trustManagerFactory.init(trustStore);
+
+ // initialize the ssl context
+ final SSLContext sslContext = SSLContext.getInstance(protocol);
+ sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), new SecureRandom());
+ return sslContext;
+ }
+
+ private static class NiFiHostnameVerifier implements HostnameVerifier {
+
+ @Override
+ public boolean verify(final String hostname, final SSLSession ssls) {
+ try {
+ for (final Certificate peerCertificate : ssls.getPeerCertificates()) {
--- End diff --
I believe this is too lenient in that it iterates over the entire certificate chain presented by the client. We are only interested in the leaf/final certificate which is the actual peer -- this should be `ssls.getPeerCertificates()[0]` (with proper edge checking, of course).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/1669
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on the issue:
https://github.com/apache/nifi/pull/1669
We may want to add some usage information to the documentation as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r111807027
--- Diff: nifi-toolkit/nifi-toolkit-admin/pom.xml ---
@@ -0,0 +1,189 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor
+license agreements. See the NOTICE file distributed with this work for additional
+information regarding copyright ownership. The ASF licenses this file to
+You under the Apache License, Version 2.0 (the "License"); you may not use
+this file except in compliance with the License. You may obtain a copy of
+the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required
+by applicable law or agreed to in writing, software distributed under the
+License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
+OF ANY KIND, either express or implied. See the License for the specific
+language governing permissions and limitations under the License. -->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-toolkit</artifactId>
+ <version>1.2.0-SNAPSHOT</version>
+ </parent>
+
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>nifi-toolkit-admin</artifactId>
+
+ <dependencies>
+ <dependency>
+ <groupId>commons-cli</groupId>
+ <artifactId>commons-cli</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-toolkit-tls</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.sun.jersey</groupId>
+ <artifactId>jersey-client</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.nifi</groupId>
+ <artifactId>nifi-client-dto</artifactId>
+ <version>1.2.0-SNAPSHOT</version>
--- End diff --
can we move the version info up into parent pom for nifi-client-dto?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by brosander <gi...@git.apache.org>.
Github user brosander commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r111808071
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/main/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientUtil.groovy ---
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.toolkit.admin.client
+
+import com.google.common.collect.Lists
+import com.sun.jersey.api.client.Client
+import com.sun.jersey.api.client.ClientResponse
+import com.sun.jersey.api.client.WebResource
+import org.apache.nifi.util.NiFiProperties
+import org.apache.nifi.util.StringUtils
+import org.apache.nifi.web.api.dto.NodeDTO
+import org.apache.nifi.web.api.entity.ClusterEntity
+import org.slf4j.Logger
+import org.slf4j.LoggerFactory
+
+public class NiFiClientUtil {
+
+ private static final Logger logger = LoggerFactory.getLogger(NiFiClientUtil.class)
+ private final static String GET_CLUSTER_ENDPOINT ="/nifi-api/controller/cluster"
+
+ public static Boolean isCluster(final NiFiProperties niFiProperties){
+ String clusterNode = niFiProperties.getProperty(NiFiProperties.CLUSTER_IS_NODE)
+ return Boolean.valueOf(clusterNode)
+ }
+
+ public static String getUrl(NiFiProperties niFiProperties, String endpoint){
+
+ final StringBuilder urlBuilder = new StringBuilder();
+
+ if(!StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT))){
+ urlBuilder.append("https://")
+ urlBuilder.append(StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_HOST)) ? "localhost": niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_HOST))
+ urlBuilder.append(":")
+ urlBuilder.append(StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT)) ? "8081" : niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT))
+ }else{
+ urlBuilder.append("http://")
+ urlBuilder.append(StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTP_HOST)) ? "localhost": niFiProperties.getProperty(NiFiProperties.WEB_HTTP_HOST))
+ urlBuilder.append(":")
+ urlBuilder.append(StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT)) ? "8080": niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT))
+ }
+
+ if(!StringUtils.isEmpty(endpoint)) {
+ urlBuilder.append(endpoint)
+ }
+
+ urlBuilder.toString()
+ }
+
+ public static String getUrl(NiFiProperties niFiProperties, NodeDTO nodeDTO, String endpoint){
+
+ final StringBuilder urlBuilder = new StringBuilder();
+ if(!StringUtils.isEmpty(niFiProperties.getProperty(NiFiProperties.WEB_HTTPS_PORT))){
+ urlBuilder.append("https://")
+
+ }else{
+ urlBuilder.append("http://")
+ }
+ urlBuilder.append(nodeDTO.address)
+ urlBuilder.append(":")
+ urlBuilder.append(nodeDTO.apiPort)
+
+ if(!StringUtils.isEmpty(endpoint)) {
+ urlBuilder.append(endpoint)
+ }
+
+ urlBuilder.toString()
+ }
+
+ public static ClusterEntity getCluster(final Client client, NiFiProperties niFiProperties, List<String> activeUrls){
+
+ if(activeUrls.isEmpty()){
+ final String url = getUrl(niFiProperties,null)
+ activeUrls.add(url)
+ }
+
+ for(String activeUrl: activeUrls) {
+
+ try {
+
+ String url = activeUrl + GET_CLUSTER_ENDPOINT
+ final WebResource webResource = client.resource(url)
+ final ClientResponse response = webResource.type("application/json").get(ClientResponse.class)
+
+ Integer status = response.getStatus()
+
+ if (status != 200) {
+ if (status == 404) {
+ logger.warn("This node is not attached to a cluster. Please connect to a node that is attached to the cluster for information")
+ } else {
+ logger.warn("Failed with HTTP error code: {}, message: {}", status, response.getStatusInfo().getReasonPhrase())
+ }
+ } else if (status == 200) {
+ return response.getEntity(ClusterEntity.class)
+ }
+
+ }catch(Exception ex){
+ logger.warn("Exception occurred during connection attempt: {}",ex.localizedMessage)
+ }
+
+ }
+
+ throw new RuntimeException("Unable to obtain cluster information")
+
+ }
+
+ public static List<String> getActiveClusterUrls(final Client client, NiFiProperties niFiProperties){
+
+ final ClusterEntity clusterEntity = getCluster(client, niFiProperties, Lists.newArrayList())
+ final List<NodeDTO> activeNodes = clusterEntity.cluster.nodes.findAll{ it.status == "CONNECTED" }
+ final List<String> activeUrls = Lists.newArrayList()
+
+ activeNodes.each {
+ activeUrls.add(getUrl(niFiProperties,it, null))
+ }
+ activeUrls
+ }
+
+ public static String convertToJson(NodeDTO nodeDTO){
--- End diff --
Can we use jackson for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r112568506
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/main/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientFactory.groovy ---
@@ -116,22 +121,27 @@ class NiFiClientFactory implements ClientFactory{
return sslContext;
}
- private static class NiFiHostnameVerifier implements HostnameVerifier {
+ static class NiFiHostnameVerifier implements HostnameVerifier {
@Override
public boolean verify(final String hostname, final SSLSession ssls) {
try {
- for (final Certificate peerCertificate : ssls.getPeerCertificates()) {
- if (peerCertificate instanceof X509Certificate) {
- final X509Certificate x509Cert = (X509Certificate) peerCertificate;
- final List<String> subjectAltNames = getSubjectAlternativeNames(x509Cert);
- if (subjectAltNames.contains(hostname.toLowerCase())) {
- return true;
- }
+
+ final Certificate peerCertificate = ssls.getPeerCertificates()[0]
+ final X509Certificate x509Cert = CertificateUtils.convertAbstractX509Certificate(peerCertificate)
+ final String dn = x509Cert.getSubjectDN().getName().trim()
+ final String cn = dn.tokenize(",").find { cn -> cn.startsWith("CN=")}
+
+ if(StringUtils.isNoneEmpty(cn) && StringUtils.substringAfter(cn,"=").toLowerCase().equals(hostname.toLowerCase())){
+ return true
+ }else {
+ final List<String> subjectAltNames = getSubjectAlternativeNames(x509Cert)
+ if (subjectAltNames.contains(hostname.toLowerCase())) {
--- End diff --
I don't know that the SANs are guaranteed to be in lowercase. However, as this is Groovy, you could use the construct
```
return getSubjectAlternativeNames(x509Cert).any { String san -> san.equalsIgnoreCase(hostname) }
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by YolandaMDavis <gi...@git.apache.org>.
Github user YolandaMDavis commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r111814074
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/groovy/org/apache/nifi/web/StandardNiFiServiceFacadeSpec.groovy ---
@@ -49,6 +53,7 @@ class StandardNiFiServiceFacadeSpec extends Specification {
SecurityContextHolder.getContext().setAuthentication(null);
}
--- End diff --
The full suite of tests were previously ignored, however I had a test that could work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on the issue:
https://github.com/apache/nifi/pull/1669
Yes. I'll merge this tonight.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1669: NIFI-3695 - created the nifi admin toolkit which includes ...
Posted by YolandaMDavis <gi...@git.apache.org>.
Github user YolandaMDavis commented on the issue:
https://github.com/apache/nifi/pull/1669
@alopresto I addressed the comments to apply LdapName as well as other minor corrections suggested
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1669: NIFI-3695 - created the nifi admin toolkit which in...
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1669#discussion_r112567879
--- Diff: nifi-toolkit/nifi-toolkit-admin/src/main/groovy/org/apache/nifi/toolkit/admin/client/NiFiClientFactory.groovy ---
@@ -116,22 +121,27 @@ class NiFiClientFactory implements ClientFactory{
return sslContext;
}
- private static class NiFiHostnameVerifier implements HostnameVerifier {
+ static class NiFiHostnameVerifier implements HostnameVerifier {
@Override
public boolean verify(final String hostname, final SSLSession ssls) {
try {
- for (final Certificate peerCertificate : ssls.getPeerCertificates()) {
- if (peerCertificate instanceof X509Certificate) {
- final X509Certificate x509Cert = (X509Certificate) peerCertificate;
- final List<String> subjectAltNames = getSubjectAlternativeNames(x509Cert);
- if (subjectAltNames.contains(hostname.toLowerCase())) {
- return true;
- }
+
+ final Certificate peerCertificate = ssls.getPeerCertificates()[0]
+ final X509Certificate x509Cert = CertificateUtils.convertAbstractX509Certificate(peerCertificate)
+ final String dn = x509Cert.getSubjectDN().getName().trim()
+ final String cn = dn.tokenize(",").find { cn -> cn.startsWith("CN=")}
--- End diff --
Unfortunately I think this could be susceptible to situations where the CN contains a `,` -- `CN=notmysite.com,mysite.com, OU=MySite, C=US` would tokenize into `CN=notmysite.com`, `mysite.com`, ` OU=MySite`, and ` C=US`.
See this [StackOverflow answer](http://stackoverflow.com/a/7933650/70465) for a code sample using `javax.naming.ldap.LdapName` to parse the DN into "RDN"s (*Relative Distinguished Name*, or the component terms of a DN).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---