You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2022/06/29 09:14:18 UTC

[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request, #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

lordgamez opened a new pull request, #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360

   https://issues.apache.org/jira/browse/MINIFICPP-1853
   
   --------------------------------------
   Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced
        in the commit message?
   
   - [ ] Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
   
   - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)?
   
   - [ ] Is your initial contribution a single, squashed commit?
   
   ### For code changes:
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the LICENSE file?
   - [ ] If applicable, have you updated the NOTICE file?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945766458


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   Updated in 67f0532544325afb6d51250209fa2f2440f95d24



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
adamdebreceni commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945746263


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   thinking about it some more, `gsl_Ensures` would document and well... ensure that some invariant is maintained, but that invariant should not come from or depend on `gsl_Ensures`, I think we should manually check if the dynamic cast succeeds, log some errors if not, and return an empty vector in that case (throwing could also work, not sure what would that trigger though)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
szaszm commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945783111


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   I think it would make more sense to document this in the return type by wrapping the shared_ptr in not_null. This would also automate the check on construction.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r940154581


##########
extensions/standard-processors/processors/GetFile.h:
##########
@@ -62,23 +62,27 @@ class GetFileMetrics : public state::response::ResponseNode {
   std::vector<state::response::SerializedResponseNode> serialize() override {
     std::vector<state::response::SerializedResponseNode> resp;
 
+    state::response::SerializedResponseNode root_node;
+    root_node.name = source_component_.getName();

Review Comment:
   I would prefer using names to make the C2 JSON more readable and the processors more recognizable, but I agree that the UUID would be currently more robust as it avoids duplicates. Do you think it should be allowed for multiple processors to share the same name?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r940164222


##########
libminifi/test/resources/TestSameProcessorMetrics.yml:
##########
@@ -0,0 +1,119 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+Flow Controller:
+    name: MiNiFi Flow
+    id: 2438e3c8-015a-1000-79ca-83af40ec1990
+Processors:
+    - name: GetFile1
+      id: 471deef6-2a6e-4a7d-912a-81cc17e3a206
+      class: org.apache.nifi.processors.standard.GetFile
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          Input Directory: /tmp/getfile1
+          Keep Source File: true
+    - name: GetFile2
+      id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
+      class: org.apache.nifi.processors.standard.GetFile
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          Input Directory: /tmp/getfile2
+          Keep Source File: true
+    - name: GetTCP1
+      id: 2438e3c8-015a-1000-79ca-83af40ec1995
+      class: org.apache.nifi.processors.standard.GetTCP
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 10 msec
+      penalization period: 30 msec
+      yield period: 10 msec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          SSL Context Service: SSLContextService
+          endpoint-list: localhost:8776
+          end-of-message-byte: d
+          reconnect-interval: 100ms
+          connection-attempt-timeout: 2000
+    - name: GetTCP2
+      id: 2438e3c8-015a-1000-79ca-83af40ec1996
+      class: org.apache.nifi.processors.standard.GetTCP
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 10 msec
+      penalization period: 30 msec
+      yield period: 10 msec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          SSL Context Service: SSLContextService
+          endpoint-list: localhost:8776
+          end-of-message-byte: d
+          reconnect-interval: 100ms
+          connection-attempt-timeout: 2000
+    - name: LogAttribute
+      id: 2438e3c8-015a-1000-79ca-83af40ec1992
+      class: org.apache.nifi.processors.standard.LogAttribute
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+        - response

Review Comment:
   Updated in 0933121cd1986fd27f20906b30f48c00d504944e



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r943169629


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -153,32 +153,34 @@ void ResponseNodeLoader::initializeFlowMonitor(const std::shared_ptr<ResponseNod
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::loadResponseNode(const std::string& clazz, core::ProcessGroup* root) {
-  auto response_node = getResponseNode(clazz);
-  if (!response_node) {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::loadResponseNodes(const std::string& clazz, core::ProcessGroup* root) {
+  auto response_nodes = getResponseNodes(clazz);
+  if (response_nodes.empty()) {
     logger_->log_error("No metric defined for %s", clazz);
-    return nullptr;
+    return {};
   }
 
-  initializeRepositoryMetrics(response_node);
-  initializeQueueMetrics(response_node, root);
-  initializeAgentIdentifier(response_node);
-  initializeAgentMonitor(response_node);
-  initializeAgentNode(response_node);
-  initializeConfigurationChecksums(response_node);
-  initializeFlowMonitor(response_node, root);
-  return response_node;
+  for (const auto& response_node : response_nodes) {
+    initializeRepositoryMetrics(response_node);
+    initializeQueueMetrics(response_node, root);
+    initializeAgentIdentifier(response_node);
+    initializeAgentMonitor(response_node);
+    initializeAgentNode(response_node);
+    initializeConfigurationChecksums(response_node);
+    initializeFlowMonitor(response_node, root);
+  }
+  return response_nodes;
 }
 
-std::shared_ptr<state::response::ResponseNode> ResponseNodeLoader::getComponentMetricsNode(const std::string& metrics_class) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getComponentMetricsNodes(const std::string& metrics_class) const {
   if (!metrics_class.empty()) {
     std::lock_guard<std::mutex> lock(component_metrics_mutex_);
-    const auto citer = component_metrics_.find(metrics_class);
-    if (citer != component_metrics_.end()) {
-      return citer->second;
+    auto it = component_metrics_.find(metrics_class);

Review Comment:
   It may have needed to be changed in some point of the development, but it is unneeded now, changed in bf46f4dbb1245cd6b1ba4108f08cfaf56694d0ae



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
fgerlits commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r942658742


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -153,32 +153,34 @@ void ResponseNodeLoader::initializeFlowMonitor(const std::shared_ptr<ResponseNod
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::loadResponseNode(const std::string& clazz, core::ProcessGroup* root) {
-  auto response_node = getResponseNode(clazz);
-  if (!response_node) {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::loadResponseNodes(const std::string& clazz, core::ProcessGroup* root) {
+  auto response_nodes = getResponseNodes(clazz);
+  if (response_nodes.empty()) {
     logger_->log_error("No metric defined for %s", clazz);
-    return nullptr;
+    return {};
   }
 
-  initializeRepositoryMetrics(response_node);
-  initializeQueueMetrics(response_node, root);
-  initializeAgentIdentifier(response_node);
-  initializeAgentMonitor(response_node);
-  initializeAgentNode(response_node);
-  initializeConfigurationChecksums(response_node);
-  initializeFlowMonitor(response_node, root);
-  return response_node;
+  for (const auto& response_node : response_nodes) {
+    initializeRepositoryMetrics(response_node);
+    initializeQueueMetrics(response_node, root);
+    initializeAgentIdentifier(response_node);
+    initializeAgentMonitor(response_node);
+    initializeAgentNode(response_node);
+    initializeConfigurationChecksums(response_node);
+    initializeFlowMonitor(response_node, root);
+  }
+  return response_nodes;
 }
 
-std::shared_ptr<state::response::ResponseNode> ResponseNodeLoader::getComponentMetricsNode(const std::string& metrics_class) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getComponentMetricsNodes(const std::string& metrics_class) const {
   if (!metrics_class.empty()) {
     std::lock_guard<std::mutex> lock(component_metrics_mutex_);
-    const auto citer = component_metrics_.find(metrics_class);
-    if (citer != component_metrics_.end()) {
-      return citer->second;
+    auto it = component_metrics_.find(metrics_class);

Review Comment:
   very minor, but why did this change from const to non-const?



##########
C2.md:
##########
@@ -126,85 +128,87 @@ The options below metrics define the sub-trees within metrics: typedmetrics and
 The classes sub option will define the metrics classes that are placed within this sub-tree. For the RESTProtocol, the above
 configuration produces the following JSON:
 
-  "metrics": {
-    "RuntimeMetrics": {
-        "deviceInfo": {
-            "systemInfo": {
-                "cpuUtilization": -1,
-                "machinearch": "x86_64",
-                "memoryUsage": 13103550464,
-                "operatingSystem": "Linux",
-                "physicalMem": 67024097280,
-                "vCores": 12
-            },
-            "networkInfo": {
-                "hostname": "ggyimesi-5540-ubuntu",
-                "ipAddress": "192.168.50.181"
-            },
-            "identifier": "13396751919892753964"
-        },
-        "flowInfo": {
-            "versionedFlowSnapshotURI": {
-                "bucketId": "default",
-                "flowId": "8db40550-db5d-11ec-95d7-0433c2c9832b"
-            },
-            "queues": {
-                "2438e3c8-015a-1000-79ca-83af40ec1997": {
-                    "dataSize": 0,
-                    "dataSizeMax": 1048576,
-                    "name": "GetFile/success/LogAttribute",
-                    "size": 0,
-                    "sizeMax": 0,
-                    "uuid": "2438e3c8-015a-1000-79ca-83af40ec1997"
-                }
-            },
-            "components": {
-                "FlowController": {
-                    "running": true,
-                    "uuid": "2438e3c8-015a-1000-79ca-83af40ec1990"
-                },
-                "GetFile": {
-                    "running": false,
-                    "uuid": "2438e3c8-015a-1000-79ca-83af40ec1991"
-                },
-                "LogAttribute": {
-                    "running": true,
-                    "uuid": "2438e3c8-015a-1000-79ca-83af40ec1992"
-                }
-            },
-            "flowId": "8db40550-db5d-11ec-95d7-0433c2c9832b"
-        }
-    },
-    "LoadMetrics": {
-        "QueueMetrics": {
-            "GetFile/success/LogAttribute": {
-                "datasize": "0",
-                "datasizemax": "1048576",
-                "queued": "0",
-                "queuedmax": "0"
-            }
-        },
-        "RepositoryMetrics": {
-            "ff": {
-                "full": false,
-                "running": false,
-                "size": "0"
-            },
-            "repo_name": {
-                "full": false,
-                "running": true,
-                "size": "0"
-            }
-        }
-    },
-    "ProcessorMetrics": {
-        "GetFileMetrics": {
-            "AcceptedFiles": 0,
-            "InputBytes": 0,
-            "OnTriggerInvocations": 0
-        }
+    "metrics": {
+      "RuntimeMetrics": {
+          "deviceInfo": {
+              "systemInfo": {
+                  "cpuUtilization": -1,
+                  "machinearch": "x86_64",
+                  "memoryUsage": 13103550464,
+                  "operatingSystem": "Linux",
+                  "physicalMem": 67024097280,
+                  "vCores": 12
+              },
+              "networkInfo": {
+                  "hostname": "ggyimesi-5540-ubuntu",
+                  "ipAddress": "192.168.50.181"
+              },
+              "identifier": "13396751919892753964"
+          },
+          "flowInfo": {
+              "versionedFlowSnapshotURI": {
+                  "bucketId": "default",
+                  "flowId": "8db40550-db5d-11ec-95d7-0433c2c9832b"
+              },
+              "queues": {
+                  "2438e3c8-015a-1000-79ca-83af40ec1997": {
+                      "dataSize": 0,
+                      "dataSizeMax": 1048576,
+                      "name": "GetFile/success/LogAttribute",
+                      "size": 0,
+                      "sizeMax": 0,
+                      "uuid": "2438e3c8-015a-1000-79ca-83af40ec1997"
+                  }
+              },
+              "components": {
+                  "FlowController": {
+                      "running": true,
+                      "uuid": "2438e3c8-015a-1000-79ca-83af40ec1990"
+                  },
+                  "GetFile": {
+                      "running": false,
+                      "uuid": "2438e3c8-015a-1000-79ca-83af40ec1991"
+                  },
+                  "LogAttribute": {
+                      "running": true,
+                      "uuid": "2438e3c8-015a-1000-79ca-83af40ec1992"
+                  }
+              },
+              "flowId": "8db40550-db5d-11ec-95d7-0433c2c9832b"
+          }
+      },
+      "LoadMetrics": {
+          "QueueMetrics": {
+              "GetFile/success/LogAttribute": {
+                  "datasize": "0",
+                  "datasizemax": "1048576",
+                  "queued": "0",
+                  "queuedmax": "0"
+              }
+          },
+          "RepositoryMetrics": {
+              "ff": {
+                  "full": false,
+                  "running": false,
+                  "size": "0"
+              },
+              "repo_name": {
+                  "full": false,
+                  "running": true,
+                  "size": "0"
+              }
+          }
+      },
+      "ProcessorMetrics": {
+          "GetFileMetrics": {
+              "GetFile1": {

Review Comment:
   should this be the UUID instead of the name of the processor?



##########
libminifi/src/c2/C2Client.cpp:
##########
@@ -191,23 +193,23 @@ std::shared_ptr<state::response::ResponseNode> C2Client::loadC2ResponseConfigura
 
 std::optional<state::response::NodeReporter::ReportedNode> C2Client::getMetricsNode(const std::string& metrics_class) const {
   std::lock_guard<std::mutex> guard{metrics_mutex_};
+  const auto createReportedNode = [](const std::vector<std::shared_ptr<state::response::ResponseNode>>& nodes) {
+    state::response::NodeReporter::ReportedNode reported_node;
+    reported_node.is_array = nodes[0]->isArray();

Review Comment:
   A `gsl_Expects(!nodes.empty())` would make this look safer.  Or maybe pass in `name` and `is_array` as parameters?



##########
C2.md:
##########
@@ -21,14 +21,17 @@ options defined are located in minifi.properties.
 
 ## Table of Contents
 
-- [Description](#description)
-- [Configuration](#configuration)
-  - [Base Options](#base-options)
-  - [Metrics](#metrics)
-  - [Protocols](#protocols)
-  - [Triggers](#triggers)
-  - [UpdatePolicies](#updatepolicies)
- - [Documentation](#documentation)
+- [Apache NiFi - MiNiFi - C++ C2 Readme.](#apache-nifi---minifi---c-c2-readme)
+  - [Table of Contents](#table-of-contents)
+  - [Description](#description)
+  - [Configuration](#configuration)
+    - [Base Options](#base-options)
+    - [Metrics](#metrics)
+- [in minifi.properties](#in-minifiproperties)

Review Comment:
   this looks like a bug in the table-of-contents-generator



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r946494835


##########
extensions/http-curl/tests/C2DescribeMetricsTest.cpp:
##########
@@ -0,0 +1,156 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include <memory>
+
+#include "TestBase.h"
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "processors/TailFile.h"
+#include "state/ProcessorController.h"
+#include "utils/file/FileUtils.h"
+#include "utils/TestUtils.h"
+#include "processors/GetTCP.h"
+#include "utils/StringUtils.h"
+#include "utils/file/PathUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class VerifyEmptyC2Metric : public VerifyC2Base {
+ public:
+  explicit VerifyEmptyC2Metric(const std::atomic_bool& metrics_found) : metrics_found_(metrics_found) {
+  }
+
+  void testSetup() override {
+    LogTestController::getInstance().setTrace<c2::C2Agent>();
+    LogTestController::getInstance().setTrace<c2::C2Client>();
+    LogTestController::getInstance().setDebug<c2::RESTSender>();
+    LogTestController::getInstance().setDebug<FlowController>();
+    LogTestController::getInstance().setOff<processors::GetTCP>();
+    VerifyC2Base::testSetup();
+  }
+
+  void runAssertions() override {
+    using org::apache::nifi::minifi::utils::verifyEventHappenedInPollTime;
+    assert(verifyEventHappenedInPollTime(40s, [&] { return metrics_found_.load(); }, 1s));
+  }
+
+ private:
+  const std::atomic_bool& metrics_found_;
+};
+
+class MetricsHandler: public HeartbeatHandler {
+ public:
+  explicit MetricsHandler(std::atomic_bool& metrics_found, std::shared_ptr<Configure> configuration)
+    : HeartbeatHandler(std::move(configuration)),
+      metrics_found_(metrics_found) {
+  }
+
+  void handleHeartbeat(const rapidjson::Document&, struct mg_connection* conn) override {
+    switch (state_) {
+      case TestState::DESCRIBE_SPECIFIC_METRIC: {
+        sendHeartbeatResponse("DESCRIBE", "metrics", "889347", conn, {{"metricsClass", "GetFileMetrics"}});
+        break;
+      }
+      case TestState::DESCRIBE_ALL_METRICS: {
+        sendHeartbeatResponse("DESCRIBE", "metrics", "889347", conn);
+        break;
+      }
+      default:
+        throw std::runtime_error("Unhandled test state");
+    }
+  }
+
+  void handleAcknowledge(const rapidjson::Document& root) override {
+    switch (state_) {
+      case TestState::DESCRIBE_SPECIFIC_METRIC: {
+        verifySpecificMetrics(root);
+        break;
+      }
+      case TestState::DESCRIBE_ALL_METRICS: {
+        verifyAllMetrics(root);
+        break;
+      }
+      default:
+        throw std::runtime_error("Unhandled test state");
+    }
+  }
+
+ private:
+  enum class TestState {
+    DESCRIBE_SPECIFIC_METRIC,
+    DESCRIBE_ALL_METRICS
+  };
+
+  static void sendEmptyHeartbeatResponse(struct mg_connection* conn) {
+    mg_printf(conn, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+  }
+
+  void verifySpecificMetrics(const rapidjson::Document& root) {
+    auto getfile_metrics_verified =
+      !root.HasMember("metrics") &&
+      root.HasMember("GetFileMetrics") &&
+      root["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a206") &&
+      root["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a207");

Review Comment:
   Good idead, updated in 1a8cb7944dced4cb1b7df52abcbbffe84ff25e4d



##########
extensions/http-curl/tests/C2MetricsTest.cpp:
##########
@@ -169,8 +169,9 @@ class MetricsHandler: public HeartbeatHandler {
 
   static bool verifyProcessorMetrics(const rapidjson::Value& processor_metrics) {
     return processor_metrics.HasMember("GetTCPMetrics") &&
-      processor_metrics["GetTCPMetrics"].HasMember("OnTriggerInvocations") &&
-      processor_metrics["GetTCPMetrics"]["OnTriggerInvocations"].GetUint() > 0;
+      processor_metrics["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1991") &&
+      processor_metrics["GetTCPMetrics"]["2438e3c8-015a-1000-79ca-83af40ec1991"].HasMember("OnTriggerInvocations") &&
+      processor_metrics["GetTCPMetrics"]["2438e3c8-015a-1000-79ca-83af40ec1991"]["OnTriggerInvocations"].GetUint() > 0;

Review Comment:
   Updated in 1a8cb7944dced4cb1b7df52abcbbffe84ff25e4d



##########
extensions/http-curl/tests/C2SameProcessorMetrics.cpp:
##########
@@ -0,0 +1,110 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include <memory>
+
+#include "TestBase.h"
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "processors/TailFile.h"
+#include "state/ProcessorController.h"
+#include "utils/file/FileUtils.h"
+#include "utils/TestUtils.h"
+#include "processors/GetTCP.h"
+#include "utils/StringUtils.h"
+#include "utils/file/PathUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class VerifyEmptyC2Metric : public VerifyC2Base {
+ public:
+  explicit VerifyEmptyC2Metric(const std::atomic_bool& metrics_found) : metrics_found_(metrics_found) {
+  }
+
+  void testSetup() override {
+    LogTestController::getInstance().setTrace<minifi::c2::C2Agent>();
+    LogTestController::getInstance().setTrace<minifi::c2::C2Client>();
+    LogTestController::getInstance().setDebug<minifi::c2::RESTSender>();
+    LogTestController::getInstance().setDebug<minifi::FlowController>();
+    LogTestController::getInstance().setOff<minifi::processors::GetTCP>();
+    VerifyC2Base::testSetup();
+  }
+
+  void runAssertions() override {
+    using org::apache::nifi::minifi::utils::verifyEventHappenedInPollTime;
+    assert(verifyEventHappenedInPollTime(40s, [&] { return metrics_found_.load(); }, 1s));
+  }
+
+ private:
+  const std::atomic_bool& metrics_found_;
+};
+
+class MetricsHandler: public HeartbeatHandler {
+ public:
+  explicit MetricsHandler(std::atomic_bool& metrics_found, std::shared_ptr<minifi::Configure> configuration)
+    : HeartbeatHandler(std::move(configuration)),
+      metrics_found_(metrics_found) {
+  }
+
+  void handleHeartbeat(const rapidjson::Document& root, struct mg_connection* conn) override {
+    verifyMetrics(root);
+    sendEmptyHeartbeatResponse(conn);
+  }
+
+ private:
+  static void sendEmptyHeartbeatResponse(struct mg_connection* conn) {
+    mg_printf(conn, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+  }
+
+  void verifyMetrics(const rapidjson::Document& root) {
+    auto initial_metrics_verified =
+      root.HasMember("metrics") &&
+      root["metrics"].HasMember("ProcessorMetrics") &&
+      root["metrics"]["ProcessorMetrics"].HasMember("GetFileMetrics") &&
+      root["metrics"]["ProcessorMetrics"]["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a206") &&
+      root["metrics"]["ProcessorMetrics"]["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a207") &&
+      root["metrics"]["ProcessorMetrics"]["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1995") &&
+      root["metrics"]["ProcessorMetrics"]["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1996");

Review Comment:
   Updated in 1a8cb7944dced4cb1b7df52abcbbffe84ff25e4d



##########
extensions/http-curl/tests/C2DescribeMetricsTest.cpp:
##########
@@ -0,0 +1,156 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include <memory>
+
+#include "TestBase.h"
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "processors/TailFile.h"
+#include "state/ProcessorController.h"
+#include "utils/file/FileUtils.h"
+#include "utils/TestUtils.h"
+#include "processors/GetTCP.h"
+#include "utils/StringUtils.h"
+#include "utils/file/PathUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class VerifyEmptyC2Metric : public VerifyC2Base {
+ public:
+  explicit VerifyEmptyC2Metric(const std::atomic_bool& metrics_found) : metrics_found_(metrics_found) {
+  }
+
+  void testSetup() override {
+    LogTestController::getInstance().setTrace<c2::C2Agent>();
+    LogTestController::getInstance().setTrace<c2::C2Client>();
+    LogTestController::getInstance().setDebug<c2::RESTSender>();
+    LogTestController::getInstance().setDebug<FlowController>();
+    LogTestController::getInstance().setOff<processors::GetTCP>();
+    VerifyC2Base::testSetup();
+  }
+
+  void runAssertions() override {
+    using org::apache::nifi::minifi::utils::verifyEventHappenedInPollTime;
+    assert(verifyEventHappenedInPollTime(40s, [&] { return metrics_found_.load(); }, 1s));
+  }
+
+ private:
+  const std::atomic_bool& metrics_found_;
+};
+
+class MetricsHandler: public HeartbeatHandler {
+ public:
+  explicit MetricsHandler(std::atomic_bool& metrics_found, std::shared_ptr<Configure> configuration)
+    : HeartbeatHandler(std::move(configuration)),
+      metrics_found_(metrics_found) {
+  }
+
+  void handleHeartbeat(const rapidjson::Document&, struct mg_connection* conn) override {
+    switch (state_) {
+      case TestState::DESCRIBE_SPECIFIC_METRIC: {
+        sendHeartbeatResponse("DESCRIBE", "metrics", "889347", conn, {{"metricsClass", "GetFileMetrics"}});
+        break;
+      }
+      case TestState::DESCRIBE_ALL_METRICS: {
+        sendHeartbeatResponse("DESCRIBE", "metrics", "889347", conn);
+        break;
+      }
+      default:
+        throw std::runtime_error("Unhandled test state");
+    }
+  }
+
+  void handleAcknowledge(const rapidjson::Document& root) override {
+    switch (state_) {
+      case TestState::DESCRIBE_SPECIFIC_METRIC: {
+        verifySpecificMetrics(root);
+        break;
+      }
+      case TestState::DESCRIBE_ALL_METRICS: {
+        verifyAllMetrics(root);
+        break;
+      }
+      default:
+        throw std::runtime_error("Unhandled test state");
+    }
+  }
+
+ private:
+  enum class TestState {
+    DESCRIBE_SPECIFIC_METRIC,
+    DESCRIBE_ALL_METRICS
+  };
+
+  static void sendEmptyHeartbeatResponse(struct mg_connection* conn) {
+    mg_printf(conn, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+  }
+
+  void verifySpecificMetrics(const rapidjson::Document& root) {
+    auto getfile_metrics_verified =
+      !root.HasMember("metrics") &&
+      root.HasMember("GetFileMetrics") &&
+      root["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a206") &&
+      root["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a207");
+    if (getfile_metrics_verified) {
+      state_ = TestState::DESCRIBE_ALL_METRICS;
+    }
+  }
+
+  void verifyAllMetrics(const rapidjson::Document& root) {
+    auto all_metrics_verified =
+      root.HasMember("metrics") &&
+      root["metrics"].HasMember("ProcessorMetrics") &&
+      root["metrics"]["ProcessorMetrics"].HasMember("GetFileMetrics") &&
+      root["metrics"]["ProcessorMetrics"]["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a206") &&
+      root["metrics"]["ProcessorMetrics"]["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a207") &&
+      root["metrics"]["ProcessorMetrics"]["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1995") &&
+      root["metrics"]["ProcessorMetrics"]["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1996") &&

Review Comment:
   Updated in 1a8cb7944dced4cb1b7df52abcbbffe84ff25e4d



##########
libminifi/include/c2/C2Client.h:
##########
@@ -77,7 +77,7 @@ class C2Client : public core::Flow, public state::response::NodeReporter {
   bool initialized_ = false;
   std::shared_ptr<core::logging::Logger> logger_;
   mutable std::mutex metrics_mutex_;
-  std::map<std::string, std::shared_ptr<state::response::ResponseNode>> root_response_nodes_;
+  std::unordered_map<std::string, std::vector<std::shared_ptr<state::response::ResponseNode>>> root_response_nodes_;

Review Comment:
   Added comment in 1a8cb7944dced4cb1b7df52abcbbffe84ff25e4d



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
adamdebreceni commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945734342


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   in that case would a `gsl_Ensures` make sense here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r943168131


##########
C2.md:
##########
@@ -21,14 +21,17 @@ options defined are located in minifi.properties.
 
 ## Table of Contents
 
-- [Description](#description)
-- [Configuration](#configuration)
-  - [Base Options](#base-options)
-  - [Metrics](#metrics)
-  - [Protocols](#protocols)
-  - [Triggers](#triggers)
-  - [UpdatePolicies](#updatepolicies)
- - [Documentation](#documentation)
+- [Apache NiFi - MiNiFi - C++ C2 Readme.](#apache-nifi---minifi---c-c2-readme)
+  - [Table of Contents](#table-of-contents)
+  - [Description](#description)
+  - [Configuration](#configuration)
+    - [Base Options](#base-options)
+    - [Metrics](#metrics)
+- [in minifi.properties](#in-minifiproperties)

Review Comment:
   Fixed in bf46f4dbb1245cd6b1ba4108f08cfaf56694d0ae



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
szaszm closed pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945824857


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   Added it as an additional TODO point in https://issues.apache.org/jira/browse/MINIFICPP-1869



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r940163923


##########
C2.md:
##########
@@ -55,48 +58,49 @@ With this change, heartbeat with agent manifest included is sent only for the fi
 light weight heartbeat. If for some reason the C2 server does not receive the first full heartbeat, the manifest can
 be requested via C2 DESCRIBE manifest command.
 
-	#in minifi.properties
+```
+#in minifi.properties
 
-	# Disable/Enable C2
-	nifi.c2.enable=true
+# Disable/Enable C2
+nifi.c2.enable=true
 

Review Comment:
   Updated in 0933121cd1986fd27f20906b30f48c00d504944e



##########
extensions/http-curl/tests/C2DescribeMetricsTest.cpp:
##########
@@ -0,0 +1,156 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include <memory>
+
+#include "TestBase.h"
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "processors/TailFile.h"
+#include "state/ProcessorController.h"
+#include "utils/file/FileUtils.h"
+#include "utils/TestUtils.h"
+#include "processors/GetTCP.h"
+#include "utils/StringUtils.h"
+#include "utils/file/PathUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class VerifyEmptyC2Metric : public VerifyC2Base {
+ public:
+  explicit VerifyEmptyC2Metric(const std::atomic_bool& metrics_found) : metrics_found_(metrics_found) {
+  }
+
+  void testSetup() override {
+    LogTestController::getInstance().setTrace<minifi::c2::C2Agent>();
+    LogTestController::getInstance().setTrace<minifi::c2::C2Client>();
+    LogTestController::getInstance().setDebug<minifi::c2::RESTSender>();
+    LogTestController::getInstance().setDebug<minifi::FlowController>();
+    LogTestController::getInstance().setOff<minifi::processors::GetTCP>();

Review Comment:
   Updated in 0933121cd1986fd27f20906b30f48c00d504944e



##########
libminifi/test/resources/TestSameProcessorMetrics.yml:
##########
@@ -0,0 +1,119 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+Flow Controller:
+    name: MiNiFi Flow
+    id: 2438e3c8-015a-1000-79ca-83af40ec1990
+Processors:
+    - name: GetFile1
+      id: 471deef6-2a6e-4a7d-912a-81cc17e3a206
+      class: org.apache.nifi.processors.standard.GetFile
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          Input Directory: /tmp/getfile1
+          Keep Source File: true
+    - name: GetFile2
+      id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
+      class: org.apache.nifi.processors.standard.GetFile
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          Input Directory: /tmp/getfile2
+          Keep Source File: true
+    - name: GetTCP1
+      id: 2438e3c8-015a-1000-79ca-83af40ec1995
+      class: org.apache.nifi.processors.standard.GetTCP
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 10 msec
+      penalization period: 30 msec
+      yield period: 10 msec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          SSL Context Service: SSLContextService
+          endpoint-list: localhost:8776
+          end-of-message-byte: d
+          reconnect-interval: 100ms
+          connection-attempt-timeout: 2000
+    - name: GetTCP2
+      id: 2438e3c8-015a-1000-79ca-83af40ec1996
+      class: org.apache.nifi.processors.standard.GetTCP
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 10 msec
+      penalization period: 30 msec
+      yield period: 10 msec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          SSL Context Service: SSLContextService
+          endpoint-list: localhost:8776
+          end-of-message-byte: d
+          reconnect-interval: 100ms
+          connection-attempt-timeout: 2000
+    - name: LogAttribute
+      id: 2438e3c8-015a-1000-79ca-83af40ec1992
+      class: org.apache.nifi.processors.standard.LogAttribute
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+        - response
+      Properties:
+        Log Level: info
+        Log Payload: true
+
+Connections:

Review Comment:
   Updated in 0933121cd1986fd27f20906b30f48c00d504944e



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r940204568


##########
extensions/standard-processors/processors/GetFile.h:
##########
@@ -62,23 +62,27 @@ class GetFileMetrics : public state::response::ResponseNode {
   std::vector<state::response::SerializedResponseNode> serialize() override {
     std::vector<state::response::SerializedResponseNode> resp;
 
+    state::response::SerializedResponseNode root_node;
+    root_node.name = source_component_.getName();

Review Comment:
   I updated to use UUID instead of processor name in 0a87993333e8e1a5754066088fdb804780f9779c



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
szaszm commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r925855542


##########
C2.md:
##########
@@ -55,48 +58,49 @@ With this change, heartbeat with agent manifest included is sent only for the fi
 light weight heartbeat. If for some reason the C2 server does not receive the first full heartbeat, the manifest can
 be requested via C2 DESCRIBE manifest command.
 
-	#in minifi.properties
+```
+#in minifi.properties
 
-	# Disable/Enable C2
-	nifi.c2.enable=true
+# Disable/Enable C2
+nifi.c2.enable=true
 

Review Comment:
   I prefer the original [classic markdown style](https://www.markdownguide.org/basic-syntax#code-blocks-1) over the github markdown extension.



##########
extensions/standard-processors/processors/GetFile.h:
##########
@@ -62,23 +62,27 @@ class GetFileMetrics : public state::response::ResponseNode {
   std::vector<state::response::SerializedResponseNode> serialize() override {
     std::vector<state::response::SerializedResponseNode> resp;
 
+    state::response::SerializedResponseNode root_node;
+    root_node.name = source_component_.getName();

Review Comment:
   Indexing components by UUID would be more robust, and would work when multiple processors of the same type share the same name. What do you think?



##########
libminifi/test/resources/TestSameProcessorMetrics.yml:
##########
@@ -0,0 +1,119 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+Flow Controller:
+    name: MiNiFi Flow
+    id: 2438e3c8-015a-1000-79ca-83af40ec1990
+Processors:
+    - name: GetFile1
+      id: 471deef6-2a6e-4a7d-912a-81cc17e3a206
+      class: org.apache.nifi.processors.standard.GetFile
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          Input Directory: /tmp/getfile1
+          Keep Source File: true
+    - name: GetFile2
+      id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
+      class: org.apache.nifi.processors.standard.GetFile
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          Input Directory: /tmp/getfile2
+          Keep Source File: true
+    - name: GetTCP1
+      id: 2438e3c8-015a-1000-79ca-83af40ec1995
+      class: org.apache.nifi.processors.standard.GetTCP
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 10 msec
+      penalization period: 30 msec
+      yield period: 10 msec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          SSL Context Service: SSLContextService
+          endpoint-list: localhost:8776
+          end-of-message-byte: d
+          reconnect-interval: 100ms
+          connection-attempt-timeout: 2000
+    - name: GetTCP2
+      id: 2438e3c8-015a-1000-79ca-83af40ec1996
+      class: org.apache.nifi.processors.standard.GetTCP
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 10 msec
+      penalization period: 30 msec
+      yield period: 10 msec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          SSL Context Service: SSLContextService
+          endpoint-list: localhost:8776
+          end-of-message-byte: d
+          reconnect-interval: 100ms
+          connection-attempt-timeout: 2000
+    - name: LogAttribute
+      id: 2438e3c8-015a-1000-79ca-83af40ec1992
+      class: org.apache.nifi.processors.standard.LogAttribute
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+        - response

Review Comment:
   No such relationship on LogAttribute. Did you mean success?



##########
extensions/http-curl/tests/C2DescribeMetricsTest.cpp:
##########
@@ -0,0 +1,156 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include <memory>
+
+#include "TestBase.h"
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "processors/TailFile.h"
+#include "state/ProcessorController.h"
+#include "utils/file/FileUtils.h"
+#include "utils/TestUtils.h"
+#include "processors/GetTCP.h"
+#include "utils/StringUtils.h"
+#include "utils/file/PathUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class VerifyEmptyC2Metric : public VerifyC2Base {
+ public:
+  explicit VerifyEmptyC2Metric(const std::atomic_bool& metrics_found) : metrics_found_(metrics_found) {
+  }
+
+  void testSetup() override {
+    LogTestController::getInstance().setTrace<minifi::c2::C2Agent>();
+    LogTestController::getInstance().setTrace<minifi::c2::C2Client>();
+    LogTestController::getInstance().setDebug<minifi::c2::RESTSender>();
+    LogTestController::getInstance().setDebug<minifi::FlowController>();
+    LogTestController::getInstance().setOff<minifi::processors::GetTCP>();

Review Comment:
   Since the test is under the minifi namespace, the `minifi::` prefix is not necessary.



##########
libminifi/test/resources/TestSameProcessorMetrics.yml:
##########
@@ -0,0 +1,119 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+Flow Controller:
+    name: MiNiFi Flow
+    id: 2438e3c8-015a-1000-79ca-83af40ec1990
+Processors:
+    - name: GetFile1
+      id: 471deef6-2a6e-4a7d-912a-81cc17e3a206
+      class: org.apache.nifi.processors.standard.GetFile
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          Input Directory: /tmp/getfile1
+          Keep Source File: true
+    - name: GetFile2
+      id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
+      class: org.apache.nifi.processors.standard.GetFile
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          Input Directory: /tmp/getfile2
+          Keep Source File: true
+    - name: GetTCP1
+      id: 2438e3c8-015a-1000-79ca-83af40ec1995
+      class: org.apache.nifi.processors.standard.GetTCP
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 10 msec
+      penalization period: 30 msec
+      yield period: 10 msec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          SSL Context Service: SSLContextService
+          endpoint-list: localhost:8776
+          end-of-message-byte: d
+          reconnect-interval: 100ms
+          connection-attempt-timeout: 2000
+    - name: GetTCP2
+      id: 2438e3c8-015a-1000-79ca-83af40ec1996
+      class: org.apache.nifi.processors.standard.GetTCP
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 10 msec
+      penalization period: 30 msec
+      yield period: 10 msec
+      run duration nanos: 0
+      auto-terminated relationships list:
+      Properties:
+          SSL Context Service: SSLContextService
+          endpoint-list: localhost:8776
+          end-of-message-byte: d
+          reconnect-interval: 100ms
+          connection-attempt-timeout: 2000
+    - name: LogAttribute
+      id: 2438e3c8-015a-1000-79ca-83af40ec1992
+      class: org.apache.nifi.processors.standard.LogAttribute
+      max concurrent tasks: 1
+      scheduling strategy: TIMER_DRIVEN
+      scheduling period: 1 sec
+      penalization period: 30 sec
+      yield period: 1 sec
+      run duration nanos: 0
+      auto-terminated relationships list:
+        - response
+      Properties:
+        Log Level: info
+        Log Payload: true
+
+Connections:

Review Comment:
   The GetTCP relationships are neither connected nor auto-terminated.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945739691


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   Updated to `gsl_Ensures` in a771342d0fb11f56cd568aeeb427d4019b1b01c6 (you're right, seems more logical in this context)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r940142053


##########
C2.md:
##########
@@ -55,48 +58,49 @@ With this change, heartbeat with agent manifest included is sent only for the fi
 light weight heartbeat. If for some reason the C2 server does not receive the first full heartbeat, the manifest can
 be requested via C2 DESCRIBE manifest command.
 
-	#in minifi.properties
+```
+#in minifi.properties
 
-	# Disable/Enable C2
-	nifi.c2.enable=true
+# Disable/Enable C2
+nifi.c2.enable=true
 

Review Comment:
   The reason of this change was that my editor did not parse the indented code block correctly, only the fenced code blocks. If they are interchangeable I would stick with the latter, just to have it look the same in all environments.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r943168406


##########
C2.md:
##########
@@ -126,85 +128,87 @@ The options below metrics define the sub-trees within metrics: typedmetrics and
 The classes sub option will define the metrics classes that are placed within this sub-tree. For the RESTProtocol, the above
 configuration produces the following JSON:
 
-  "metrics": {
-    "RuntimeMetrics": {
-        "deviceInfo": {
-            "systemInfo": {
-                "cpuUtilization": -1,
-                "machinearch": "x86_64",
-                "memoryUsage": 13103550464,
-                "operatingSystem": "Linux",
-                "physicalMem": 67024097280,
-                "vCores": 12
-            },
-            "networkInfo": {
-                "hostname": "ggyimesi-5540-ubuntu",
-                "ipAddress": "192.168.50.181"
-            },
-            "identifier": "13396751919892753964"
-        },
-        "flowInfo": {
-            "versionedFlowSnapshotURI": {
-                "bucketId": "default",
-                "flowId": "8db40550-db5d-11ec-95d7-0433c2c9832b"
-            },
-            "queues": {
-                "2438e3c8-015a-1000-79ca-83af40ec1997": {
-                    "dataSize": 0,
-                    "dataSizeMax": 1048576,
-                    "name": "GetFile/success/LogAttribute",
-                    "size": 0,
-                    "sizeMax": 0,
-                    "uuid": "2438e3c8-015a-1000-79ca-83af40ec1997"
-                }
-            },
-            "components": {
-                "FlowController": {
-                    "running": true,
-                    "uuid": "2438e3c8-015a-1000-79ca-83af40ec1990"
-                },
-                "GetFile": {
-                    "running": false,
-                    "uuid": "2438e3c8-015a-1000-79ca-83af40ec1991"
-                },
-                "LogAttribute": {
-                    "running": true,
-                    "uuid": "2438e3c8-015a-1000-79ca-83af40ec1992"
-                }
-            },
-            "flowId": "8db40550-db5d-11ec-95d7-0433c2c9832b"
-        }
-    },
-    "LoadMetrics": {
-        "QueueMetrics": {
-            "GetFile/success/LogAttribute": {
-                "datasize": "0",
-                "datasizemax": "1048576",
-                "queued": "0",
-                "queuedmax": "0"
-            }
-        },
-        "RepositoryMetrics": {
-            "ff": {
-                "full": false,
-                "running": false,
-                "size": "0"
-            },
-            "repo_name": {
-                "full": false,
-                "running": true,
-                "size": "0"
-            }
-        }
-    },
-    "ProcessorMetrics": {
-        "GetFileMetrics": {
-            "AcceptedFiles": 0,
-            "InputBytes": 0,
-            "OnTriggerInvocations": 0
-        }
+    "metrics": {
+      "RuntimeMetrics": {
+          "deviceInfo": {
+              "systemInfo": {
+                  "cpuUtilization": -1,
+                  "machinearch": "x86_64",
+                  "memoryUsage": 13103550464,
+                  "operatingSystem": "Linux",
+                  "physicalMem": 67024097280,
+                  "vCores": 12
+              },
+              "networkInfo": {
+                  "hostname": "ggyimesi-5540-ubuntu",
+                  "ipAddress": "192.168.50.181"
+              },
+              "identifier": "13396751919892753964"
+          },
+          "flowInfo": {
+              "versionedFlowSnapshotURI": {
+                  "bucketId": "default",
+                  "flowId": "8db40550-db5d-11ec-95d7-0433c2c9832b"
+              },
+              "queues": {
+                  "2438e3c8-015a-1000-79ca-83af40ec1997": {
+                      "dataSize": 0,
+                      "dataSizeMax": 1048576,
+                      "name": "GetFile/success/LogAttribute",
+                      "size": 0,
+                      "sizeMax": 0,
+                      "uuid": "2438e3c8-015a-1000-79ca-83af40ec1997"
+                  }
+              },
+              "components": {
+                  "FlowController": {
+                      "running": true,
+                      "uuid": "2438e3c8-015a-1000-79ca-83af40ec1990"
+                  },
+                  "GetFile": {
+                      "running": false,
+                      "uuid": "2438e3c8-015a-1000-79ca-83af40ec1991"
+                  },
+                  "LogAttribute": {
+                      "running": true,
+                      "uuid": "2438e3c8-015a-1000-79ca-83af40ec1992"
+                  }
+              },
+              "flowId": "8db40550-db5d-11ec-95d7-0433c2c9832b"
+          }
+      },
+      "LoadMetrics": {
+          "QueueMetrics": {
+              "GetFile/success/LogAttribute": {
+                  "datasize": "0",
+                  "datasizemax": "1048576",
+                  "queued": "0",
+                  "queuedmax": "0"
+              }
+          },
+          "RepositoryMetrics": {
+              "ff": {
+                  "full": false,
+                  "running": false,
+                  "size": "0"
+              },
+              "repo_name": {
+                  "full": false,
+                  "running": true,
+                  "size": "0"
+              }
+          }
+      },
+      "ProcessorMetrics": {
+          "GetFileMetrics": {
+              "GetFile1": {

Review Comment:
   Good catch, I forgot to update the documentation, updated in bf46f4dbb1245cd6b1ba4108f08cfaf56694d0ae



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r943168857


##########
libminifi/src/c2/C2Client.cpp:
##########
@@ -191,23 +193,23 @@ std::shared_ptr<state::response::ResponseNode> C2Client::loadC2ResponseConfigura
 
 std::optional<state::response::NodeReporter::ReportedNode> C2Client::getMetricsNode(const std::string& metrics_class) const {
   std::lock_guard<std::mutex> guard{metrics_mutex_};
+  const auto createReportedNode = [](const std::vector<std::shared_ptr<state::response::ResponseNode>>& nodes) {
+    state::response::NodeReporter::ReportedNode reported_node;
+    reported_node.is_array = nodes[0]->isArray();

Review Comment:
   I think using `gsl_Expects` is a good idea here, added in bf46f4dbb1245cd6b1ba4108f08cfaf56694d0ae



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
adamdebreceni commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945693234


##########
libminifi/src/c2/C2Client.cpp:
##########
@@ -218,38 +222,45 @@ std::vector<state::response::NodeReporter::ReportedNode> C2Client::getHeartbeatN
   configuration_->get(minifi::Configuration::nifi_c2_full_heartbeat, fullHb);
   const bool include = include_manifest || fullHb == "true";
 
-  std::vector<state::response::NodeReporter::ReportedNode> nodes;
+  std::vector<state::response::NodeReporter::ReportedNode> reported_nodes;
   std::lock_guard<std::mutex> guard{metrics_mutex_};
-  nodes.reserve(root_response_nodes_.size());
-  for (const auto &entry : root_response_nodes_) {
-    auto identifier = std::dynamic_pointer_cast<state::response::AgentIdentifier>(entry.second);
-    if (identifier) {
-      identifier->includeAgentManifest(include);
-    }
-    if (entry.second) {
-      state::response::NodeReporter::ReportedNode reported_node;
-      reported_node.name = entry.second->getName();
-      reported_node.is_array = entry.second->isArray();
-      reported_node.serialized_nodes = entry.second->serialize();
-      nodes.push_back(reported_node);
+  reported_nodes.reserve(root_response_nodes_.size());
+  for (const auto& [name, node_values] : root_response_nodes_) {
+    for (const auto& node : node_values) {
+      auto identifier = std::dynamic_pointer_cast<state::response::AgentIdentifier>(node);
+      if (identifier) {
+        identifier->includeAgentManifest(include);
+      }
+      if (node) {
+        state::response::NodeReporter::ReportedNode reported_node;
+        reported_node.name = node->getName();
+        reported_node.is_array = node->isArray();
+        reported_node.serialized_nodes = node->serialize();
+        reported_nodes.push_back(reported_node);
+      }
     }
   }
-  return nodes;
+  return reported_nodes;
 }
 
 void C2Client::initializeResponseNodes(core::ProcessGroup* root) {
+  if (!root_response_nodes_.empty()) {
+    return;
+  }
   std::string class_csv;
   std::lock_guard<std::mutex> guard{metrics_mutex_};
   if (configuration_->get(minifi::Configuration::nifi_c2_root_classes, class_csv)) {
     std::vector<std::string> classes = utils::StringUtils::split(class_csv, ",");
 
     for (const std::string& clazz : classes) {
-      auto response_node = response_node_loader_.loadResponseNode(clazz, root);
-      if (!response_node) {
+      auto response_nodes = response_node_loader_.loadResponseNodes(clazz, root);
+      if (response_nodes.empty()) {
         continue;
       }
 
-      root_response_nodes_[response_node->getName()] = std::move(response_node);
+      for (auto response_node: response_nodes) {
+        root_response_nodes_[response_node->getName()].push_back(std::move(response_node));

Review Comment:
   should we use `auto&&` here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
szaszm commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945794575


##########
libminifi/include/c2/C2Client.h:
##########
@@ -77,7 +77,7 @@ class C2Client : public core::Flow, public state::response::NodeReporter {
   bool initialized_ = false;
   std::shared_ptr<core::logging::Logger> logger_;
   mutable std::mutex metrics_mutex_;
-  std::map<std::string, std::shared_ptr<state::response::ResponseNode>> root_response_nodes_;
+  std::unordered_map<std::string, std::vector<std::shared_ptr<state::response::ResponseNode>>> root_response_nodes_;

Review Comment:
   Could you add a comment about the meaning of the string key, or the structure of the whole type? The type info describes a structure, but not the meaning of each part, and it just got even more complex than before.
   What keys, what sets of value nodes?



##########
extensions/http-curl/tests/C2MetricsTest.cpp:
##########
@@ -169,8 +169,9 @@ class MetricsHandler: public HeartbeatHandler {
 
   static bool verifyProcessorMetrics(const rapidjson::Value& processor_metrics) {
     return processor_metrics.HasMember("GetTCPMetrics") &&
-      processor_metrics["GetTCPMetrics"].HasMember("OnTriggerInvocations") &&
-      processor_metrics["GetTCPMetrics"]["OnTriggerInvocations"].GetUint() > 0;
+      processor_metrics["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1991") &&
+      processor_metrics["GetTCPMetrics"]["2438e3c8-015a-1000-79ca-83af40ec1991"].HasMember("OnTriggerInvocations") &&
+      processor_metrics["GetTCPMetrics"]["2438e3c8-015a-1000-79ca-83af40ec1991"]["OnTriggerInvocations"].GetUint() > 0;

Review Comment:
   Same here



##########
extensions/http-curl/tests/C2DescribeMetricsTest.cpp:
##########
@@ -0,0 +1,156 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include <memory>
+
+#include "TestBase.h"
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "processors/TailFile.h"
+#include "state/ProcessorController.h"
+#include "utils/file/FileUtils.h"
+#include "utils/TestUtils.h"
+#include "processors/GetTCP.h"
+#include "utils/StringUtils.h"
+#include "utils/file/PathUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class VerifyEmptyC2Metric : public VerifyC2Base {
+ public:
+  explicit VerifyEmptyC2Metric(const std::atomic_bool& metrics_found) : metrics_found_(metrics_found) {
+  }
+
+  void testSetup() override {
+    LogTestController::getInstance().setTrace<c2::C2Agent>();
+    LogTestController::getInstance().setTrace<c2::C2Client>();
+    LogTestController::getInstance().setDebug<c2::RESTSender>();
+    LogTestController::getInstance().setDebug<FlowController>();
+    LogTestController::getInstance().setOff<processors::GetTCP>();
+    VerifyC2Base::testSetup();
+  }
+
+  void runAssertions() override {
+    using org::apache::nifi::minifi::utils::verifyEventHappenedInPollTime;
+    assert(verifyEventHappenedInPollTime(40s, [&] { return metrics_found_.load(); }, 1s));
+  }
+
+ private:
+  const std::atomic_bool& metrics_found_;
+};
+
+class MetricsHandler: public HeartbeatHandler {
+ public:
+  explicit MetricsHandler(std::atomic_bool& metrics_found, std::shared_ptr<Configure> configuration)
+    : HeartbeatHandler(std::move(configuration)),
+      metrics_found_(metrics_found) {
+  }
+
+  void handleHeartbeat(const rapidjson::Document&, struct mg_connection* conn) override {
+    switch (state_) {
+      case TestState::DESCRIBE_SPECIFIC_METRIC: {
+        sendHeartbeatResponse("DESCRIBE", "metrics", "889347", conn, {{"metricsClass", "GetFileMetrics"}});
+        break;
+      }
+      case TestState::DESCRIBE_ALL_METRICS: {
+        sendHeartbeatResponse("DESCRIBE", "metrics", "889347", conn);
+        break;
+      }
+      default:
+        throw std::runtime_error("Unhandled test state");
+    }
+  }
+
+  void handleAcknowledge(const rapidjson::Document& root) override {
+    switch (state_) {
+      case TestState::DESCRIBE_SPECIFIC_METRIC: {
+        verifySpecificMetrics(root);
+        break;
+      }
+      case TestState::DESCRIBE_ALL_METRICS: {
+        verifyAllMetrics(root);
+        break;
+      }
+      default:
+        throw std::runtime_error("Unhandled test state");
+    }
+  }
+
+ private:
+  enum class TestState {
+    DESCRIBE_SPECIFIC_METRIC,
+    DESCRIBE_ALL_METRICS
+  };
+
+  static void sendEmptyHeartbeatResponse(struct mg_connection* conn) {
+    mg_printf(conn, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+  }
+
+  void verifySpecificMetrics(const rapidjson::Document& root) {
+    auto getfile_metrics_verified =
+      !root.HasMember("metrics") &&
+      root.HasMember("GetFileMetrics") &&
+      root["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a206") &&
+      root["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a207");
+    if (getfile_metrics_verified) {
+      state_ = TestState::DESCRIBE_ALL_METRICS;
+    }
+  }
+
+  void verifyAllMetrics(const rapidjson::Document& root) {
+    auto all_metrics_verified =
+      root.HasMember("metrics") &&
+      root["metrics"].HasMember("ProcessorMetrics") &&
+      root["metrics"]["ProcessorMetrics"].HasMember("GetFileMetrics") &&
+      root["metrics"]["ProcessorMetrics"]["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a206") &&
+      root["metrics"]["ProcessorMetrics"]["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a207") &&
+      root["metrics"]["ProcessorMetrics"]["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1995") &&
+      root["metrics"]["ProcessorMetrics"]["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1996") &&

Review Comment:
   and here



##########
extensions/http-curl/tests/C2DescribeMetricsTest.cpp:
##########
@@ -0,0 +1,156 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include <memory>
+
+#include "TestBase.h"
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "processors/TailFile.h"
+#include "state/ProcessorController.h"
+#include "utils/file/FileUtils.h"
+#include "utils/TestUtils.h"
+#include "processors/GetTCP.h"
+#include "utils/StringUtils.h"
+#include "utils/file/PathUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class VerifyEmptyC2Metric : public VerifyC2Base {
+ public:
+  explicit VerifyEmptyC2Metric(const std::atomic_bool& metrics_found) : metrics_found_(metrics_found) {
+  }
+
+  void testSetup() override {
+    LogTestController::getInstance().setTrace<c2::C2Agent>();
+    LogTestController::getInstance().setTrace<c2::C2Client>();
+    LogTestController::getInstance().setDebug<c2::RESTSender>();
+    LogTestController::getInstance().setDebug<FlowController>();
+    LogTestController::getInstance().setOff<processors::GetTCP>();
+    VerifyC2Base::testSetup();
+  }
+
+  void runAssertions() override {
+    using org::apache::nifi::minifi::utils::verifyEventHappenedInPollTime;
+    assert(verifyEventHappenedInPollTime(40s, [&] { return metrics_found_.load(); }, 1s));
+  }
+
+ private:
+  const std::atomic_bool& metrics_found_;
+};
+
+class MetricsHandler: public HeartbeatHandler {
+ public:
+  explicit MetricsHandler(std::atomic_bool& metrics_found, std::shared_ptr<Configure> configuration)
+    : HeartbeatHandler(std::move(configuration)),
+      metrics_found_(metrics_found) {
+  }
+
+  void handleHeartbeat(const rapidjson::Document&, struct mg_connection* conn) override {
+    switch (state_) {
+      case TestState::DESCRIBE_SPECIFIC_METRIC: {
+        sendHeartbeatResponse("DESCRIBE", "metrics", "889347", conn, {{"metricsClass", "GetFileMetrics"}});
+        break;
+      }
+      case TestState::DESCRIBE_ALL_METRICS: {
+        sendHeartbeatResponse("DESCRIBE", "metrics", "889347", conn);
+        break;
+      }
+      default:
+        throw std::runtime_error("Unhandled test state");
+    }
+  }
+
+  void handleAcknowledge(const rapidjson::Document& root) override {
+    switch (state_) {
+      case TestState::DESCRIBE_SPECIFIC_METRIC: {
+        verifySpecificMetrics(root);
+        break;
+      }
+      case TestState::DESCRIBE_ALL_METRICS: {
+        verifyAllMetrics(root);
+        break;
+      }
+      default:
+        throw std::runtime_error("Unhandled test state");
+    }
+  }
+
+ private:
+  enum class TestState {
+    DESCRIBE_SPECIFIC_METRIC,
+    DESCRIBE_ALL_METRICS
+  };
+
+  static void sendEmptyHeartbeatResponse(struct mg_connection* conn) {
+    mg_printf(conn, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+  }
+
+  void verifySpecificMetrics(const rapidjson::Document& root) {
+    auto getfile_metrics_verified =
+      !root.HasMember("metrics") &&
+      root.HasMember("GetFileMetrics") &&
+      root["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a206") &&
+      root["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a207");

Review Comment:
   Would you mind extracting these constrants, and giving them names like `getfile1_uuid`, `getfile2_uuid`, `gettcp1_uuid`, `gettcp2_uuid`?



##########
extensions/http-curl/tests/C2SameProcessorMetrics.cpp:
##########
@@ -0,0 +1,110 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#undef NDEBUG
+#include <memory>
+
+#include "TestBase.h"
+#include "HTTPIntegrationBase.h"
+#include "HTTPHandlers.h"
+#include "processors/TailFile.h"
+#include "state/ProcessorController.h"
+#include "utils/file/FileUtils.h"
+#include "utils/TestUtils.h"
+#include "processors/GetTCP.h"
+#include "utils/StringUtils.h"
+#include "utils/file/PathUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class VerifyEmptyC2Metric : public VerifyC2Base {
+ public:
+  explicit VerifyEmptyC2Metric(const std::atomic_bool& metrics_found) : metrics_found_(metrics_found) {
+  }
+
+  void testSetup() override {
+    LogTestController::getInstance().setTrace<minifi::c2::C2Agent>();
+    LogTestController::getInstance().setTrace<minifi::c2::C2Client>();
+    LogTestController::getInstance().setDebug<minifi::c2::RESTSender>();
+    LogTestController::getInstance().setDebug<minifi::FlowController>();
+    LogTestController::getInstance().setOff<minifi::processors::GetTCP>();
+    VerifyC2Base::testSetup();
+  }
+
+  void runAssertions() override {
+    using org::apache::nifi::minifi::utils::verifyEventHappenedInPollTime;
+    assert(verifyEventHappenedInPollTime(40s, [&] { return metrics_found_.load(); }, 1s));
+  }
+
+ private:
+  const std::atomic_bool& metrics_found_;
+};
+
+class MetricsHandler: public HeartbeatHandler {
+ public:
+  explicit MetricsHandler(std::atomic_bool& metrics_found, std::shared_ptr<minifi::Configure> configuration)
+    : HeartbeatHandler(std::move(configuration)),
+      metrics_found_(metrics_found) {
+  }
+
+  void handleHeartbeat(const rapidjson::Document& root, struct mg_connection* conn) override {
+    verifyMetrics(root);
+    sendEmptyHeartbeatResponse(conn);
+  }
+
+ private:
+  static void sendEmptyHeartbeatResponse(struct mg_connection* conn) {
+    mg_printf(conn, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 0\r\nConnection: close\r\n\r\n");
+  }
+
+  void verifyMetrics(const rapidjson::Document& root) {
+    auto initial_metrics_verified =
+      root.HasMember("metrics") &&
+      root["metrics"].HasMember("ProcessorMetrics") &&
+      root["metrics"]["ProcessorMetrics"].HasMember("GetFileMetrics") &&
+      root["metrics"]["ProcessorMetrics"]["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a206") &&
+      root["metrics"]["ProcessorMetrics"]["GetFileMetrics"].HasMember("471deef6-2a6e-4a7d-912a-81cc17e3a207") &&
+      root["metrics"]["ProcessorMetrics"]["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1995") &&
+      root["metrics"]["ProcessorMetrics"]["GetTCPMetrics"].HasMember("2438e3c8-015a-1000-79ca-83af40ec1996");

Review Comment:
   same here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r940190287


##########
extensions/standard-processors/processors/GetFile.h:
##########
@@ -62,23 +62,27 @@ class GetFileMetrics : public state::response::ResponseNode {
   std::vector<state::response::SerializedResponseNode> serialize() override {
     std::vector<state::response::SerializedResponseNode> resp;
 
+    state::response::SerializedResponseNode root_node;
+    root_node.name = source_component_.getName();

Review Comment:
   But I suppose if NiFi allows multiple processors to share the same name then we should allow it as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r940142053


##########
C2.md:
##########
@@ -55,48 +58,49 @@ With this change, heartbeat with agent manifest included is sent only for the fi
 light weight heartbeat. If for some reason the C2 server does not receive the first full heartbeat, the manifest can
 be requested via C2 DESCRIBE manifest command.
 
-	#in minifi.properties
+```
+#in minifi.properties
 
-	# Disable/Enable C2
-	nifi.c2.enable=true
+# Disable/Enable C2
+nifi.c2.enable=true
 

Review Comment:
   The reason of this change was that my editor did not parse the indented code block correctly, only the fenced code blocks. If they are interchangeable I would stick with the latter, just to have it look the same in all environments.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945735509


##########
libminifi/src/c2/C2Client.cpp:
##########
@@ -218,38 +222,45 @@ std::vector<state::response::NodeReporter::ReportedNode> C2Client::getHeartbeatN
   configuration_->get(minifi::Configuration::nifi_c2_full_heartbeat, fullHb);
   const bool include = include_manifest || fullHb == "true";
 
-  std::vector<state::response::NodeReporter::ReportedNode> nodes;
+  std::vector<state::response::NodeReporter::ReportedNode> reported_nodes;
   std::lock_guard<std::mutex> guard{metrics_mutex_};
-  nodes.reserve(root_response_nodes_.size());
-  for (const auto &entry : root_response_nodes_) {
-    auto identifier = std::dynamic_pointer_cast<state::response::AgentIdentifier>(entry.second);
-    if (identifier) {
-      identifier->includeAgentManifest(include);
-    }
-    if (entry.second) {
-      state::response::NodeReporter::ReportedNode reported_node;
-      reported_node.name = entry.second->getName();
-      reported_node.is_array = entry.second->isArray();
-      reported_node.serialized_nodes = entry.second->serialize();
-      nodes.push_back(reported_node);
+  reported_nodes.reserve(root_response_nodes_.size());
+  for (const auto& [name, node_values] : root_response_nodes_) {
+    for (const auto& node : node_values) {
+      auto identifier = std::dynamic_pointer_cast<state::response::AgentIdentifier>(node);
+      if (identifier) {
+        identifier->includeAgentManifest(include);
+      }
+      if (node) {
+        state::response::NodeReporter::ReportedNode reported_node;
+        reported_node.name = node->getName();
+        reported_node.is_array = node->isArray();
+        reported_node.serialized_nodes = node->serialize();
+        reported_nodes.push_back(reported_node);
+      }
     }
   }
-  return nodes;
+  return reported_nodes;
 }
 
 void C2Client::initializeResponseNodes(core::ProcessGroup* root) {
+  if (!root_response_nodes_.empty()) {
+    return;
+  }
   std::string class_csv;
   std::lock_guard<std::mutex> guard{metrics_mutex_};
   if (configuration_->get(minifi::Configuration::nifi_c2_root_classes, class_csv)) {
     std::vector<std::string> classes = utils::StringUtils::split(class_csv, ",");
 
     for (const std::string& clazz : classes) {
-      auto response_node = response_node_loader_.loadResponseNode(clazz, root);
-      if (!response_node) {
+      auto response_nodes = response_node_loader_.loadResponseNodes(clazz, root);
+      if (response_nodes.empty()) {
         continue;
       }
 
-      root_response_nodes_[response_node->getName()] = std::move(response_node);
+      for (auto response_node: response_nodes) {
+        root_response_nodes_[response_node->getName()].push_back(std::move(response_node));

Review Comment:
   Updated in 567764bbdc4bac04b07b50337293ae732fd8580d



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945736178


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   Added a `gsl_Expects` for this in 567764bbdc4bac04b07b50337293ae732fd8580d



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
adamdebreceni commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945697728


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   is "having only non-null elements" an invariant of the return value of `getResponseNodes`, or do we check everywhere we use it for nulls?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945731158


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   In my experience it's more of an invariant as it is passed through a lot of other functions, casting it in a few places as well, and in some places we check for nullptr and in other places we don't.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1360: MINIFICPP-1853 Enable multiple metrics from the same processor type

Posted by GitBox <gi...@apache.org>.
lordgamez commented on code in PR #1360:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1360#discussion_r945822803


##########
libminifi/src/core/state/nodes/ResponseNodeLoader.cpp:
##########
@@ -59,17 +59,17 @@ void ResponseNodeLoader::initializeComponentMetrics(core::ProcessGroup* root) {
     node_source->getResponseNodes(metric_vector);
     std::lock_guard<std::mutex> guard(component_metrics_mutex_);
     for (const auto& metric : metric_vector) {
-      component_metrics_[metric->getName()] = metric;
+      component_metrics_[metric->getName()].push_back(metric);
     }
   }
 }
 
-std::shared_ptr<ResponseNode> ResponseNodeLoader::getResponseNode(const std::string& clazz) const {
+std::vector<std::shared_ptr<ResponseNode>> ResponseNodeLoader::getResponseNodes(const std::string& clazz) const {
   std::shared_ptr<core::CoreComponent> ptr = core::ClassLoader::getDefaultClassLoader().instantiate(clazz, clazz);
   if (ptr == nullptr) {
-    return getComponentMetricsNode(clazz);
+    return getComponentMetricsNodes(clazz);
   }
-  return std::dynamic_pointer_cast<ResponseNode>(ptr);
+  return {std::dynamic_pointer_cast<ResponseNode>(ptr)};

Review Comment:
   As I checked that would be a bigger refactor, as this is the main source of the response nodes, that change would propagate through `MetricsBase`, `ResponseNodeLoader`, C2Client and so on everywhere where `std::shared_ptr<state::response::ResponseNode>` is used for parameters and class members. I would keep this manual check for the moment.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org