You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by achristianson <gi...@git.apache.org> on 2018/05/31 16:04:22 UTC
[GitHub] nifi-minifi-cpp pull request #350: Minificpp 465
GitHub user achristianson opened a pull request:
https://github.com/apache/nifi-minifi-cpp/pull/350
Minificpp 465
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with MINIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] If applicable, have you updated the LICENSE file?
- [x] If applicable, have you updated the NOTICE file?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-465
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi-minifi-cpp/pull/350.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #350
----
commit d2a6d0e8dda528dd8e98812c0d284390196616e2
Author: Andrew I. Christianson <an...@...>
Date: 2018-05-29T17:08:56Z
MINIFIPP-514 Incorporated regex property validation information into agent manifest
commit ab01f42b3140d998879434fa6317d28b6e72f2d2
Author: Andrew I. Christianson <an...@...>
Date: 2018-05-31T16:03:50Z
MINIFICPP-465 Implemented regex validation of properties
----
---
[GitHub] nifi-minifi-cpp pull request #350: MINIFICPP-465 Implemented regex validatio...
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/350#discussion_r192165462
--- Diff: libminifi/src/core/yaml/YamlConfiguration.cpp ---
@@ -782,29 +783,59 @@ void YamlConfiguration::parsePropertiesNodeYaml(YAML::Node *propertiesNode,
}
}
+ validateComponentProperties(processor, component_name, yaml_section);
+}
+
+void YamlConfiguration::validateComponentProperties(const std::shared_ptr<ConfigurableComponent> &component,
+ const std::string &component_name,
+ const std::string &yaml_section) const {
+ const auto &component_properties = component->getProperties();
+
// Validate required properties
- for (const auto &prop_pair : processor->getProperties()) {
+ for (const auto &prop_pair : component_properties) {
if (prop_pair.second.getRequired()) {
- const auto &val = prop_pair.second.getValue();
-
- if (val.empty()) {
- // Build a helpful error message for the user so they can fix the
- // invalid YAML config file, using the component name if present
- std::string err_msg =
- "Unable to parse configuration file for component named '"
- + component_name
- + "' because required property '" + prop_pair.second.getName() + "' is not set";
- if (!yaml_section.empty()) {
- err_msg += " [in '" + yaml_section + "' section of configuration file]";
- }
- logging::LOG_ERROR(logger_) << err_msg;
+ if (prop_pair.second.getValue().empty()) {
+ std::string reason("required property '");
+ reason.append(prop_pair.second.getName());
+ reason.append("' is not set");
+ raiseComponentError(component_name, yaml_section, reason);
+ }
+ }
+ }
- throw std::invalid_argument(err_msg);
+ // Validate regex properties
+ for (const auto &prop_pair : component_properties) {
+ const auto &prop_regex_str = prop_pair.second.getValidRegex();
+
+ if (!prop_regex_str.empty()) {
+ std::regex prop_regex(prop_regex_str);
+ if (!std::regex_match(prop_pair.second.getValue(), prop_regex)) {
+ std::string reason("property '");
+ reason.append(prop_pair.second.getName());
+ reason.append("' does not match validation pattern '");
+ reason.append(prop_regex_str);
+ reason.append("'");
+ raiseComponentError(component_name, yaml_section, reason);
}
}
}
}
+void YamlConfiguration::raiseComponentError(const std::string &component_name,
+ const std::string &yaml_section,
+ const std::string &reason) const {
+ std::string err_msg = "Unable to parse configuration file for component named '";
--- End diff --
The logger implements <<, so why not utilize that instead of calling append?
---
[GitHub] nifi-minifi-cpp pull request #350: MINIFICPP-465 Implemented regex validatio...
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/350#discussion_r192165204
--- Diff: libminifi/src/core/yaml/YamlConfiguration.cpp ---
@@ -782,29 +783,59 @@ void YamlConfiguration::parsePropertiesNodeYaml(YAML::Node *propertiesNode,
}
}
+ validateComponentProperties(processor, component_name, yaml_section);
+}
+
+void YamlConfiguration::validateComponentProperties(const std::shared_ptr<ConfigurableComponent> &component,
+ const std::string &component_name,
+ const std::string &yaml_section) const {
+ const auto &component_properties = component->getProperties();
+
// Validate required properties
- for (const auto &prop_pair : processor->getProperties()) {
+ for (const auto &prop_pair : component_properties) {
if (prop_pair.second.getRequired()) {
- const auto &val = prop_pair.second.getValue();
-
- if (val.empty()) {
- // Build a helpful error message for the user so they can fix the
- // invalid YAML config file, using the component name if present
- std::string err_msg =
- "Unable to parse configuration file for component named '"
- + component_name
- + "' because required property '" + prop_pair.second.getName() + "' is not set";
- if (!yaml_section.empty()) {
- err_msg += " [in '" + yaml_section + "' section of configuration file]";
- }
- logging::LOG_ERROR(logger_) << err_msg;
+ if (prop_pair.second.getValue().empty()) {
+ std::string reason("required property '");
--- End diff --
Why not use a string stream here?
---
[GitHub] nifi-minifi-cpp pull request #350: MINIFICPP-465 Implemented regex validatio...
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/350#discussion_r192166370
--- Diff: libminifi/src/core/yaml/YamlConfiguration.cpp ---
@@ -782,29 +783,59 @@ void YamlConfiguration::parsePropertiesNodeYaml(YAML::Node *propertiesNode,
}
}
+ validateComponentProperties(processor, component_name, yaml_section);
+}
+
+void YamlConfiguration::validateComponentProperties(const std::shared_ptr<ConfigurableComponent> &component,
+ const std::string &component_name,
+ const std::string &yaml_section) const {
+ const auto &component_properties = component->getProperties();
+
// Validate required properties
- for (const auto &prop_pair : processor->getProperties()) {
+ for (const auto &prop_pair : component_properties) {
if (prop_pair.second.getRequired()) {
- const auto &val = prop_pair.second.getValue();
-
- if (val.empty()) {
- // Build a helpful error message for the user so they can fix the
- // invalid YAML config file, using the component name if present
- std::string err_msg =
- "Unable to parse configuration file for component named '"
- + component_name
- + "' because required property '" + prop_pair.second.getName() + "' is not set";
- if (!yaml_section.empty()) {
- err_msg += " [in '" + yaml_section + "' section of configuration file]";
- }
- logging::LOG_ERROR(logger_) << err_msg;
+ if (prop_pair.second.getValue().empty()) {
+ std::string reason("required property '");
+ reason.append(prop_pair.second.getName());
+ reason.append("' is not set");
+ raiseComponentError(component_name, yaml_section, reason);
+ }
+ }
+ }
- throw std::invalid_argument(err_msg);
+ // Validate regex properties
+ for (const auto &prop_pair : component_properties) {
+ const auto &prop_regex_str = prop_pair.second.getValidRegex();
+
+ if (!prop_regex_str.empty()) {
+ std::regex prop_regex(prop_regex_str);
+ if (!std::regex_match(prop_pair.second.getValue(), prop_regex)) {
+ std::string reason("property '");
--- End diff --
Same comment RE the readability. Stringstream may help with making this easier to read.
---
[GitHub] nifi-minifi-cpp issue #350: MINIFICPP-465 Implemented regex validation of pr...
Posted by achristianson <gi...@git.apache.org>.
Github user achristianson commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/350
Fixed.
---
[GitHub] nifi-minifi-cpp pull request #350: MINIFICPP-465 Implemented regex validatio...
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/350#discussion_r192165818
--- Diff: libminifi/test/unit/YamlConfigurationTests.cpp ---
@@ -457,3 +456,64 @@ Flow Controller:
REQUIRE(NULL != rootFlowConfig->findProcessor("XYZ")->getUUID());
REQUIRE(!rootFlowConfig->findProcessor("XYZ")->getUUIDStr().empty());
}
+
+class DummyComponent : public core::ConfigurableComponent {
--- End diff --
let's leave override out of libminifi/ and keep it in extensions due to compiler versioning.
---
[GitHub] nifi-minifi-cpp issue #350: MINIFICPP-465 Implemented regex validation of pr...
Posted by achristianson <gi...@git.apache.org>.
Github user achristianson commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/350
Build fails are timeouts after 52/53 passed unit tests; suspect possible travis environmental issue.
---
[GitHub] nifi-minifi-cpp pull request #350: MINIFICPP-465 Implemented regex validatio...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi-minifi-cpp/pull/350
---
[GitHub] nifi-minifi-cpp issue #350: MINIFICPP-465 Implemented regex validation of pr...
Posted by achristianson <gi...@git.apache.org>.
Github user achristianson commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/350
Rebased & all issues addressed. Ready for review.
---
[GitHub] nifi-minifi-cpp pull request #350: MINIFICPP-465 Implemented regex validatio...
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/350#discussion_r192165690
--- Diff: libminifi/src/core/yaml/YamlConfiguration.cpp ---
@@ -782,29 +783,59 @@ void YamlConfiguration::parsePropertiesNodeYaml(YAML::Node *propertiesNode,
}
}
+ validateComponentProperties(processor, component_name, yaml_section);
+}
+
+void YamlConfiguration::validateComponentProperties(const std::shared_ptr<ConfigurableComponent> &component,
+ const std::string &component_name,
+ const std::string &yaml_section) const {
+ const auto &component_properties = component->getProperties();
+
// Validate required properties
- for (const auto &prop_pair : processor->getProperties()) {
+ for (const auto &prop_pair : component_properties) {
if (prop_pair.second.getRequired()) {
- const auto &val = prop_pair.second.getValue();
-
- if (val.empty()) {
- // Build a helpful error message for the user so they can fix the
- // invalid YAML config file, using the component name if present
- std::string err_msg =
- "Unable to parse configuration file for component named '"
- + component_name
- + "' because required property '" + prop_pair.second.getName() + "' is not set";
- if (!yaml_section.empty()) {
- err_msg += " [in '" + yaml_section + "' section of configuration file]";
- }
- logging::LOG_ERROR(logger_) << err_msg;
+ if (prop_pair.second.getValue().empty()) {
+ std::string reason("required property '");
+ reason.append(prop_pair.second.getName());
+ reason.append("' is not set");
+ raiseComponentError(component_name, yaml_section, reason);
+ }
+ }
+ }
- throw std::invalid_argument(err_msg);
+ // Validate regex properties
+ for (const auto &prop_pair : component_properties) {
+ const auto &prop_regex_str = prop_pair.second.getValidRegex();
+
+ if (!prop_regex_str.empty()) {
+ std::regex prop_regex(prop_regex_str);
+ if (!std::regex_match(prop_pair.second.getValue(), prop_regex)) {
+ std::string reason("property '");
+ reason.append(prop_pair.second.getName());
+ reason.append("' does not match validation pattern '");
+ reason.append(prop_regex_str);
+ reason.append("'");
+ raiseComponentError(component_name, yaml_section, reason);
}
}
}
}
+void YamlConfiguration::raiseComponentError(const std::string &component_name,
+ const std::string &yaml_section,
+ const std::string &reason) const {
+ std::string err_msg = "Unable to parse configuration file for component named '";
--- End diff --
A string stream or the like may make this more readable.
---
[GitHub] nifi-minifi-cpp pull request #350: MINIFICPP-465 Implemented regex validatio...
Posted by phrocker <gi...@git.apache.org>.
Github user phrocker commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/350#discussion_r192164378
--- Diff: libminifi/src/core/yaml/YamlConfiguration.cpp ---
@@ -19,6 +19,7 @@
#include <memory>
#include <vector>
#include <set>
+#include <regex>
--- End diff --
Not all versions of the compiler support <regex> You'll need to handle the C and C++ paths. I think GetFile is an example.
---