You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by benqiu2016 <gi...@git.apache.org> on 2017/01/23 17:21:15 UTC
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
GitHub user benqiu2016 opened a pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41
MINIFI-184: Add Security Support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/benqiu2016/nifi-minifi-cpp master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi-minifi-cpp/pull/41.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #41
----
commit dc96f2ad7dce8a6934516d4485ed92f38caf3de4
Author: Bin Qiu <be...@gmail.com>
Date: 2016-10-28T16:48:57Z
Provenance Support
commit b0ee74792dd67b61d7e09d6abfb7d4eb05dfc2df
Author: Bin Qiu <be...@gmail.com>
Date: 2016-10-28T16:53:20Z
Provenance Support
commit 351fab8f9a7b629ec16986f54e909e34e2cf0b1e
Author: Bin Qiu <be...@gmail.com>
Date: 2016-11-14T17:21:31Z
MINIFI-131: Provenance Support
commit 4e6d1eec360bebd0b63422159ea1ef17b088b879
Author: Bin Qiu <be...@gmail.com>
Date: 2016-11-14T17:26:21Z
MINIFI-131: Provenance Support
commit 836e4f3993aed01a647327b20212569527b66dbf
Author: Bin Qiu <be...@gmail.com>
Date: 2016-11-23T04:51:12Z
MINIFI-114: Set default stateFile for TailFile
commit c41021ddb837b9db414ed03817aefb4d6f070a58
Author: Bin Qiu <be...@gmail.com>
Date: 2017-01-03T05:02:51Z
sync head
commit c1279d8063ad9ad469065e63d1c028878d787d97
Author: Bin Qiu <be...@gmail.com>
Date: 2017-01-22T18:10:34Z
MINIFI-184: Add Security Support
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by benqiu2016 <gi...@git.apache.org>.
Github user benqiu2016 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99379332
--- Diff: cmake/FindOpenSSL.cmake ---
@@ -0,0 +1,28 @@
+# OPENSSL_ROOT_DIR - Set this variable to the root installation of OpenSSL
--- End diff --
if i do not provide that, build failed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by benqiu2016 <gi...@git.apache.org>.
Github user benqiu2016 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99883760
--- Diff: cmake/FindOpenSSL.cmake ---
@@ -0,0 +1,28 @@
+# OPENSSL_ROOT_DIR - Set this variable to the root installation of OpenSSL
+#
+# Read-Only variables:
+# OPENSSL_FOUND - system has the OpenSSL library
+# OPENSSL_INCLUDE_DIR - the OpenSSL include directory
+# OPENSSL_LIBRARIES - The libraries needed to use OpenSSL
+# OPENSSL_VERSION - This is set to $major.$minor.$revision$path (eg. 0.9.8s)
+
+find_path(OPENSSL_INCLUDE_DIR
+ NAMES openssl/ssl.h
--- End diff --
openssl is used pretty widely and very stable and they provide bug fix regularly to plug the security hole.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/41
reviewing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98260232
--- Diff: libminifi/src/Configure.cpp ---
@@ -29,6 +29,12 @@ const char *Configure::nifi_server_report_interval= "nifi.server.report.interval
const char *Configure::nifi_provenance_repository_max_storage_size = "nifi.provenance.repository.max.storage.size";
const char *Configure::nifi_provenance_repository_max_storage_time = "nifi.provenance.repository.max.storage.time";
const char *Configure::nifi_provenance_repository_directory_default = "nifi.provenance.repository.directory.default";
+const char *Configure::nifi_remote_input_secure = "nifi.remote.input.secure";
+const char *Configure::nifi_security_need_ClientAuth = "nifi.security.need.ClientAuth";
+const char *Configure::nifi_security_client_certificate = "nifi.security.client.certificate";
+const char *Configure::nifi_security_client_private_key = "nifi.security.client.private.key";
+const char *Configure::nifi_security_client_pass_phase = "nifi.security.client.pass.phase";
--- End diff --
likely should just be passphrase
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98259124
--- Diff: README.md ---
@@ -243,6 +245,26 @@ Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - sc
Port: 10001
Host Name: localhost
+### Site2Site Security Configuration
+
+ in minifi.properties
+
+ enable tls ssl
+ nifi.remote.input.secure=true
+
+ if you want to enable client certificate base authorization
+ nifi.security.need.ClientAuth=true
+ setup the client certificate and private key PEM files
+ nifi.security.client.certificate=./conf/client.pem
+ nifi.security.client.private.key=./conf/client.pem
+ setup the client private key passphase file
+ nifi.security.client.pass.phase=./conf/password
+ setup the client CA certificate file
+ nif.security.client.ca.certificate=./conf/nifi-cert.pem
--- End diff --
typo for nifi
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99357721
--- Diff: libminifi/src/FlowController.cpp ---
@@ -75,22 +78,123 @@ FlowController::FlowController(std::string name)
}
}
- path = realpath(adjustedFilename.c_str(), full_path);
+ path = realpath(adjustedFilename.c_str(), full_path);
if (!path)
{
_logger->log_error("Could not locate path from provided configuration file name (%s). Exiting.", full_path);
exit(1);
}
- std::string pathString(path);
- _configurationFileName = pathString;
- _logger->log_info("FlowController NiFi Configuration file %s", pathString.c_str());
+ std::string pathString(path);
--- End diff --
Would it make sense to break this out into a separate class to reduce coupling with the security library within the FlowController?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi-minifi-cpp/pull/41
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/41
Changes look good. I saw there were a couple of inconsistencies with some of the docs and typos but corrected those and will incorporate in the merge. Thanks for the fixes!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/41
reviewing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98476667
--- Diff: libminifi/src/Site2SitePeer.cpp ---
@@ -237,12 +271,31 @@ int Site2SitePeer::readData(uint8_t *buf, int buflen, CRC32 *crc)
Close();
return status;
}
- status = recv(_socket, buf, buflen, 0);
- if (status <= 0)
+ if (!_ssl)
{
- Close();
- // this->yield();
- return status;
+ status = recv(_socket, buf, buflen, 0);
+ if (status <= 0)
+ {
+ Close();
+ // this->yield();
+ return status;
+ }
+ }
+ else
+ {
+ // for SSL, wait for the TLS IO is completed
+ int sslStatus;
+ do {
+ status = SSL_read(_ssl, buf, buflen);
+ sslStatus = SSL_get_error(_ssl, status);
+ }
+ while (status < 0 && sslStatus == SSL_ERROR_WANT_READ);
+ if (status <= 0)
+ {
+ Close();
+ // this->yield();
--- End diff --
should this be removed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98476688
--- Diff: libminifi/src/Site2SitePeer.cpp ---
@@ -237,12 +271,31 @@ int Site2SitePeer::readData(uint8_t *buf, int buflen, CRC32 *crc)
Close();
return status;
}
- status = recv(_socket, buf, buflen, 0);
- if (status <= 0)
+ if (!_ssl)
{
- Close();
- // this->yield();
- return status;
+ status = recv(_socket, buf, buflen, 0);
+ if (status <= 0)
+ {
+ Close();
+ // this->yield();
--- End diff --
should this be removed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99357485
--- Diff: cmake/FindOpenSSL.cmake ---
@@ -0,0 +1,28 @@
+# OPENSSL_ROOT_DIR - Set this variable to the root installation of OpenSSL
+#
+# Read-Only variables:
+# OPENSSL_FOUND - system has the OpenSSL library
+# OPENSSL_INCLUDE_DIR - the OpenSSL include directory
+# OPENSSL_LIBRARIES - The libraries needed to use OpenSSL
+# OPENSSL_VERSION - This is set to $major.$minor.$revision$path (eg. 0.9.8s)
+
+find_path(OPENSSL_INCLUDE_DIR
+ NAMES openssl/ssl.h
--- End diff --
Did you consider BOTAN? That's a purely C++/Crypto library ( one of many ). Any particular reason we are introducing a C library when C++ libraries exist?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99357571
--- Diff: libminifi/include/FlowController.h ---
@@ -90,6 +93,25 @@ class FlowController
}
return _flowController;
}
+ //! passphase for the private file callback
--- End diff --
Could we extract SSL Functionality into separate classes so do we don't couple the controller with security?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98259383
--- Diff: libminifi/src/Configure.cpp ---
@@ -29,6 +29,12 @@ const char *Configure::nifi_server_report_interval= "nifi.server.report.interval
const char *Configure::nifi_provenance_repository_max_storage_size = "nifi.provenance.repository.max.storage.size";
const char *Configure::nifi_provenance_repository_max_storage_time = "nifi.provenance.repository.max.storage.time";
const char *Configure::nifi_provenance_repository_directory_default = "nifi.provenance.repository.directory.default";
+const char *Configure::nifi_remote_input_secure = "nifi.remote.input.secure";
+const char *Configure::nifi_security_need_ClientAuth = "nifi.security.need.ClientAuth";
--- End diff --
if we are matching nifi's, this should be nifi.security.needClientAuth
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98517708
--- Diff: README.md ---
@@ -243,6 +245,26 @@ Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - sc
Port: 10001
Host Name: localhost
+### Site2Site Security Configuration
+
+ in minifi.properties
--- End diff --
We should stub these out as defaults in minifi.properties. No values, but at least provide a quick way of having them be readily available.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by benqiu2016 <gi...@git.apache.org>.
Github user benqiu2016 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99884147
--- Diff: libminifi/src/FlowController.cpp ---
@@ -75,22 +78,123 @@ FlowController::FlowController(std::string name)
}
}
- path = realpath(adjustedFilename.c_str(), full_path);
+ path = realpath(adjustedFilename.c_str(), full_path);
if (!path)
{
_logger->log_error("Could not locate path from provided configuration file name (%s). Exiting.", full_path);
exit(1);
}
- std::string pathString(path);
- _configurationFileName = pathString;
- _logger->log_info("FlowController NiFi Configuration file %s", pathString.c_str());
+ std::string pathString(path);
--- End diff --
Same as above
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98259083
--- Diff: cmake/FindOpenSSL.cmake ---
@@ -0,0 +1,28 @@
+# OPENSSL_ROOT_DIR - Set this variable to the root installation of OpenSSL
--- End diff --
I believe this should be unneeded as cmake has a built in FindOpenSSL module.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #41: MINIFI-184: Add Security Support
Posted by benqiu2016 <gi...@git.apache.org>.
Github user benqiu2016 commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/41
@phrocker @apiri remove the FindOpenSSL cmake by setting the OPENSSL_ROOT_DIR. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98259141
--- Diff: libminifi/src/Configure.cpp ---
@@ -29,6 +29,12 @@ const char *Configure::nifi_server_report_interval= "nifi.server.report.interval
const char *Configure::nifi_provenance_repository_max_storage_size = "nifi.provenance.repository.max.storage.size";
const char *Configure::nifi_provenance_repository_max_storage_time = "nifi.provenance.repository.max.storage.time";
const char *Configure::nifi_provenance_repository_directory_default = "nifi.provenance.repository.directory.default";
+const char *Configure::nifi_remote_input_secure = "nifi.remote.input.secure";
+const char *Configure::nifi_security_need_ClientAuth = "nifi.security.need.ClientAuth";
+const char *Configure::nifi_security_client_certificate = "nifi.security.client.certificate";
+const char *Configure::nifi_security_client_private_key = "nifi.security.client.private.key";
+const char *Configure::nifi_security_client_pass_phase = "nifi.security.client.pass.phase";
+const char *Configure::nifi_security_client_ca_certificate = "nif.security.client.ca.certificate";
--- End diff --
typo for nifi
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by benqiu2016 <gi...@git.apache.org>.
Github user benqiu2016 commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r99884121
--- Diff: libminifi/include/FlowController.h ---
@@ -90,6 +93,25 @@ class FlowController
}
return _flowController;
}
+ //! passphase for the private file callback
--- End diff --
All the central service like provenance and others are tied with flowController like current NiFi.
It make code more clear.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp issue #41: MINIFI-184: Add Security Support
Posted by benqiu2016 <gi...@git.apache.org>.
Github user benqiu2016 commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/41
@apiri new code commit to address your review comment, back out the IOS port.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi-minifi-cpp pull request #41: MINIFI-184: Add Security Support
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/41#discussion_r98260259
--- Diff: README.md ---
@@ -243,6 +245,26 @@ Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - sc
Port: 10001
Host Name: localhost
+### Site2Site Security Configuration
+
+ in minifi.properties
+
+ enable tls ssl
+ nifi.remote.input.secure=true
+
+ if you want to enable client certificate base authorization
+ nifi.security.need.ClientAuth=true
+ setup the client certificate and private key PEM files
+ nifi.security.client.certificate=./conf/client.pem
+ nifi.security.client.private.key=./conf/client.pem
+ setup the client private key passphase file
+ nifi.security.client.pass.phase=./conf/password
--- End diff --
likely should just be passphrase
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---