You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Adam Hunyadi (Jira)" <ji...@apache.org> on 2021/02/09 12:23:00 UTC
[jira] [Resolved] (MINIFICPP-1421) Investigate and fix C2JstackTest
[ https://issues.apache.org/jira/browse/MINIFICPP-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Hunyadi resolved MINIFICPP-1421.
-------------------------------------
Assignee: Adam Hunyadi
Resolution: Fixed
> Investigate and fix C2JstackTest
> --------------------------------
>
> Key: MINIFICPP-1421
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1421
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Adam Hunyadi
> Assignee: Adam Hunyadi
> Priority: Minor
> Labels: MiNiFi-CPP-Hygiene
> Fix For: 0.9.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> *Background:*
> C2JstackTest is currently looks for a log line that contains the word "SchedulingAgent". This line however is only present due to the LogTestController logs that log lines from the SchedulingAgents will not be shown.
> {code:bash|title=Filtered test output}
> ➜ ./extensions/http-curl/tests/C2JstackTest ../src/libminifi/test/resources/TestHTTPGet.yml ../src/libminifi/test/resources/ |& grep "SchedulingAgent"
> [2020-12-08 10:48:51.955] [org::apache::nifi::minifi::core::logging::LoggerConfiguration] [debug] org::apache::nifi::minifi::SchedulingAgent logger got sinks from namespace root and level error from namespace root
> [2020-12-08 10:48:51.955] [org::apache::nifi::minifi::core::logging::LoggerConfiguration] [debug] org::apache::nifi::minifi::ThreadedSchedulingAgent logger got sinks from namespace root and level error from namespace root
> [2020-12-08 10:48:51.955] [org::apache::nifi::minifi::core::logging::LoggerConfiguration] [debug] org::apache::nifi::minifi::TimerDrivenSchedulingAgent logger got sinks from namespace root and level error from namespace root
> {code}
> Enabling all sinks still did not produce any relevant log line that would be worth matching against:
> {code:c++}
> #include "../../extensions/standard-processors/controllers/UnorderedMapKeyValueStoreService.h"
> #include "../../extensions/standard-processors/controllers/UnorderedMapPersistableKeyValueStoreService.h"
> #include "../../extensions/http-curl/processors/InvokeHTTP.h"
> #include "../../extensions/http-curl/processors/InvokeHTTP.h"
> #include "../../extensions/standard-processors/processors/LogAttribute.h"
> #include "../../extensions/standard-processors/processors/LogAttribute.h"
> #include "../../libminifi/include/c2/ControllerSocketProtocol.h"
> #include "../../libminifi/include/c2/ControllerSocketProtocol.h"
> #include "../../extensions/standard-processors/processors/AppendHostInfo.h"
> #include "../../extensions/standard-processors/processors/AppendHostInfo.h"
> #include "../../extensions/standard-processors/processors/ExecuteProcess.h"
> #include "../../extensions/standard-processors/processors/ExecuteProcess.h"
> #include "../../extensions/standard-processors/processors/ExtractText.h"
> #include "../../extensions/standard-processors/processors/ExtractText.h"
> #include "../../extensions/standard-processors/processors/GenerateFlowFile.h"
> #include "../../extensions/standard-processors/processors/GenerateFlowFile.h"
> #include "../../extensions/standard-processors/processors/GetFile.h"
> #include "../../extensions/standard-processors/processors/GetFile.h"
> #include "../../extensions/standard-processors/processors/GetTCP.h"
> #include "../../extensions/standard-processors/processors/GetTCP.h"
> #include "../../extensions/standard-processors/processors/HashContent.h"
> #include "../../extensions/standard-processors/processors/HashContent.h"
> #include "../../extensions/standard-processors/processors/ListenSyslog.h"
> #include "../../extensions/standard-processors/processors/ListenSyslog.h"
> #include "../../extensions/standard-processors/processors/PutFile.h"
> #include "../../extensions/standard-processors/processors/PutFile.h"
> #include "../../extensions/standard-processors/processors/RetryFlowFile.h"
> #include "../../extensions/standard-processors/processors/RetryFlowFile.h"
> #include "../../extensions/standard-processors/processors/RouteOnAttribute.h"
> #include "../../extensions/standard-processors/processors/RouteOnAttribute.h"
> #include "../../extensions/standard-processors/processors/TailFile.h"
> #include "../../extensions/standard-processors/processors/TailFile.h"
> #include "../../extensions/standard-processors/processors/UpdateAttribute.h"
> #include "../../extensions/standard-processors/processors/UpdateAttribute.h"
> #include "../../extensions/http-curl/protocols/AgentPrinter.h"
> #include "../../extensions/http-curl/protocols/AgentPrinter.h"
> #include "../../extensions/http-curl/protocols/RESTReceiver.h"
> #include "../../extensions/http-curl/protocols/RESTReceiver.h"
> #include "../../extensions/civetweb/processors/ListenHTTP.h"
> #include "../../extensions/civetweb/processors/ListenHTTP.h"
> (...)
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::utils::IdGenerator>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::FlowFile>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::provenance::ProvenanceEventRecord>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::FlowFileRecord>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::Properties>();
> LogTestController::getInstance().setTrace<LogTestController>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::c2::C2Agent>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::c2::RESTSender>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::FlowController>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::utils::HTTPClient>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::Connectable>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::Repository>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::repository::VolatileContentRepository>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::utils::file::FileSystem>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::FlowConfiguration>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::ConfigurableComponent>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::controller::StandardControllerServiceProvider>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::YamlConfiguration>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::controllers::AbstractAutoPersistingKeyValueStoreService>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::controllers::UnorderedMapKeyValueStoreService>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::controllers::UnorderedMapPersistableKeyValueStoreService>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::controller::StandardControllerServiceNode>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::ProcessGroup>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::Processor>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::InvokeHTTP>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::LogAttribute>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::Connection>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::c2::C2Client>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::FlowControlProtocol>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::SchedulingAgent>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::ThreadedSchedulingAgent>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::TimerDrivenSchedulingAgent>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::repository::FileSystemRepository>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::ProcessContext>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::core::ProcessSession>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::provenance::ProvenanceReporter>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::utils::ByteOutputCallback>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::c2::ControllerSocketProtocol>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::AppendHostInfo>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::ExecuteProcess>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::ExtractText>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::GenerateFlowFile>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::GetFile>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::GetTCP>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::HashContent>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::ListenSyslog>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::PutFile>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::RetryFlowFile>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::RouteOnAttribute>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::TailFile>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::UpdateAttribute>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::c2::AgentPrinter>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::c2::RESTReceiver>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::processors::ListenHTTP>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::controllers::SSLContextService>();
> LogTestController::getInstance().setTrace<org::apache::nifi::minifi::controllers::UpdatePolicyControllerService>();
> {code}
> {code:bash|title=Test output with filters}
> ➜ build ./extensions/http-curl/tests/C2JstackTest ../src/libminifi/test/resources/TestHTTPGet.yml ../src/libminifi/test/resources/ |& egrep -v "got sinks|log level" | grep "SchedulingAgent"
> [2020-12-08 13:23:08.954] [org::apache::nifi::minifi::ThreadedSchedulingAgent] [debug] Shutting down threads for processor invoke/2438e3c8-015a-1000-79ca-83af40ec1991
> [2020-12-08 13:23:08.954] [org::apache::nifi::minifi::ThreadedSchedulingAgent] [debug] Shutting down threads for processor invoke/2438e3c8-015a-1000-79ca-83af40ec1991
> [2020-12-08 13:23:08.954] [org::apache::nifi::minifi::ThreadedSchedulingAgent] [warning] Cannot unschedule threads for processor invoke because it is not running
> [2020-12-08 13:23:08.954] [org::apache::nifi::minifi::ThreadedSchedulingAgent] [debug] Shutting down threads for processor LogAttribute/2438e3c8-015a-1000-79ca-83af40ec1992
> {code}
> *Proposal:*
> As the test breaks if once we disable the inclusion of "logger got sinks" by default on test, we should turn off this test and at some point investigate what exactly the test was meant to check. One could probably check out the version of the MiNiFi this was introduced in and see if something important was captured by checking against this log line.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)