You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by ad...@apache.org on 2025/08/11 14:23:45 UTC
(nifi-minifi-cpp) branch minifi-api-reduced updated (6932d420a -> 75cdd4966)
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a change to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
omit 6932d420a Fix parameter name
omit f0e774155 Linter fix, rollback temp test changes
omit 1791048eb Linter fix
omit e1034aa78 Build fix
omit 23cad31c8 Review changes
omit f16ce262f Review changes
omit 0b85be8c0 Win fix
omit e27e50970 Fix linux build
omit f1e118a9e Review changes
omit 121a1e606 Provide the correct name on processor construction
omit 7e77dce02 Windows fix
omit 8b3deded5 Rebase fix
omit 74d3f841c Windows proc update
omit 457904371 Review changes, upgrade concurrentqueue
omit e1f58e202 Update extensions/python/ExecutePythonProcessor.cpp
omit d5f0883b0 Rename
omit c4eeed2ae Add dtor, review changes
omit 770253cf0 Fix llamacpp proc/test
omit 88f9a60f5 Remove obsolete file
omit 84c248963 Rebase fix
omit 765fcf50a Linter fixes
omit f408c1165 Linter fixes
omit fd32d58d5 Linter fix
omit 3f1d61f10 Build fix
omit 0aff9e5e5 Remove logger
omit 99a39f049 Rebase fix
omit ecfd69446 Rebase fix
omit dd88a015c Rebase fix
omit 9c75eb12a Rebase fix
omit 29261732b Rebase fix
omit e24a8b9c8 Trigger ci
omit 95c6025e9 Rebase fix
omit 54b48f24a Review changes
omit 0a61c4989 Rebase fix
omit 90e3148ac Tests work
omit efa9bda30 Working on api reduction
add a473f11a9 MINIFICPP-2597 Replace removed thirdparty actions
add ef701211b MINIFICPP-2591 Add new value option to FetchOPCProcessor lazy mode
add e3474f98e MINIFICPP-2598 Fixing inherited ControllerService manifest issues
new 5be79f51e Working on api reduction
new b0fb553b9 Tests work
new 7be07f990 Rebase fix
new a28b89e7a Review changes
new ff2985505 Rebase fix
new b7d3b52fe Trigger ci
new f1ce1feb1 Rebase fix
new c04a6d89a Rebase fix
new b208d843c Rebase fix
new ff80ed560 Rebase fix
new 13460b685 Rebase fix
new a57cfec50 Remove logger
new d635fda25 Build fix
new 02f016bb6 Linter fix
new 4bbe474db Linter fixes
new 59b524ba2 Linter fixes
new 6a5532264 Rebase fix
new 34559a0fb Remove obsolete file
new 13bda7c0d Fix llamacpp proc/test
new fc75710b3 Add dtor, review changes
new c4cd764c1 Rename
new 6dbbb6cfd Update extensions/python/ExecutePythonProcessor.cpp
new bf1134e5a Review changes, upgrade concurrentqueue
new 3c3264d65 Windows proc update
new f5cd565c7 Rebase fix
new eed06552a Windows fix
new b4645cbb4 Provide the correct name on processor construction
new bf75a8ede Review changes
new c236217bc Fix linux build
new a1a4cbd87 Win fix
new b43f26bc2 Review changes
new 443827570 Review changes
new c0dfe3a9e Build fix
new a96d25102 Linter fix
new ce6080e8e Linter fix, rollback temp test changes
new 567aa7606 Fix parameter name
new 75cdd4966 Fix rebase
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6932d420a)
\
N -- N -- N refs/heads/minifi-api-reduced (75cdd4966)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 37 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/ci.yml | 4 +-
PROCESSORS.md | 30 ++--
controller/MiNiFiController.cpp | 25 ++--
controller/tests/ControllerTests.cpp | 8 +-
core-framework/include/agent/agent_docs.h | 3 +-
.../include/core/controller/ControllerService.h | 4 +
core-framework/include/http/BaseHTTPClient.h | 6 +-
core-framework/include/http/HTTPClient.h | 15 +-
core-framework/include/utils/net/AsioSocketUtils.h | 21 ++-
core-framework/src/http/HTTPClient.cpp | 4 +-
core-framework/src/utils/net/AsioSocketUtils.cpp | 6 +-
.../include/controllers/SSLContextService.h | 2 +-
.../controllers/SSLContextServiceInterface.h | 4 +-
extension-framework/src/utils/net/Ssl.cpp | 6 +-
extensions/civetweb/tests/ListenHTTPTests.cpp | 6 +-
.../controllerservices/CouchbaseClusterService.cpp | 8 +-
.../controllerservices/CouchbaseClusterService.h | 6 +-
extensions/elasticsearch/PostElasticsearch.cpp | 4 +-
extensions/elasticsearch/PostElasticsearch.h | 4 +-
extensions/grafana-loki/PushGrafanaLoki.cpp | 6 +-
extensions/grafana-loki/PushGrafanaLoki.h | 6 +-
extensions/kafka/KafkaProcessorBase.cpp | 2 +-
extensions/kafka/KafkaProcessorBase.h | 4 +-
extensions/kafka/PublishKafka.h | 4 +-
extensions/opc/include/fetchopc.h | 48 +++++--
extensions/opc/include/putopc.h | 4 +-
extensions/opc/src/fetchopc.cpp | 101 +++++++++++---
extensions/opc/tests/FetchOpcProcessorTests.cpp | 95 +++++++++++++
extensions/opc/tests/OpcUaTestServer.h | 30 +++-
extensions/prometheus/PrometheusExposerWrapper.h | 4 +-
extensions/python/ExecutePythonProcessor.cpp | 4 +-
extensions/python/types/PyProcessContext.cpp | 2 +-
extensions/python/types/PySSLContextService.h | 4 +-
extensions/sftp/processors/PutSFTP.h | 2 +-
extensions/sftp/processors/SFTPProcessorBase.h | 2 +-
extensions/splunk/PutSplunkHTTP.h | 2 +-
extensions/splunk/SplunkHECProcessor.cpp | 6 +-
extensions/splunk/SplunkHECProcessor.h | 10 +-
.../controllers/JsonRecordSetWriter.h | 1 +
.../controllers/JsonTreeReader.h | 1 +
.../standard-processors/modbus/FetchModbusTcp.cpp | 2 +-
.../standard-processors/modbus/FetchModbusTcp.h | 4 +-
.../standard-processors/processors/GetTCP.cpp | 2 +-
extensions/standard-processors/processors/GetTCP.h | 4 +-
.../standard-processors/processors/InvokeHTTP.cpp | 2 +-
.../standard-processors/processors/InvokeHTTP.h | 6 +-
.../processors/ListenSyslog.cpp | 3 +-
.../standard-processors/processors/ListenSyslog.h | 4 +-
.../standard-processors/processors/ListenTCP.cpp | 4 +-
.../standard-processors/processors/ListenTCP.h | 8 +-
.../standard-processors/processors/ListenUDP.cpp | 2 +-
.../standard-processors/processors/PutTCP.cpp | 2 +-
extensions/standard-processors/processors/PutTCP.h | 16 ++-
.../tests/integration/VerifyInvokeHTTP.h | 2 +-
.../standard-processors/tests/unit/GetTCPTests.cpp | 6 +-
.../tests/unit/ListenSyslogTests.cpp | 6 +-
.../tests/unit/ListenTcpTests.cpp | 16 +--
.../tests/unit/ManifestTests.cpp | 152 +++++++++++++++++++++
.../standard-processors/tests/unit/PutTCPTests.cpp | 8 +-
libminifi/include/RemoteProcessorGroupPort.h | 4 +-
libminifi/include/c2/ControllerSocketProtocol.h | 6 +-
libminifi/include/c2/protocols/RESTSender.h | 4 +-
.../controllers/NetworkPrioritizerService.h | 1 -
libminifi/include/controllers/SSLContextService.h | 32 +++--
libminifi/include/core/logging/alert/AlertSink.h | 4 +-
libminifi/include/sitetosite/SiteToSite.h | 8 +-
libminifi/include/sitetosite/SiteToSiteClient.h | 4 +-
libminifi/src/RemoteProcessorGroupPort.cpp | 22 ++-
libminifi/src/c2/ControllerSocketProtocol.cpp | 27 ++--
libminifi/src/c2/protocols/RESTSender.cpp | 6 +-
libminifi/src/controllers/SSLContextService.cpp | 40 +++---
libminifi/src/core/logging/alert/AlertSink.cpp | 4 +-
.../src/core/state/nodes/AgentInformation.cpp | 13 +-
.../ControllerServiceIntegrationTests.cpp | 16 +--
libminifi/test/integration/HTTPSiteToSiteTests.cpp | 2 +-
libminifi/test/integration/SiteToSiteRestTest.cpp | 15 +-
.../integration/TimeoutHTTPSiteToSiteTests.cpp | 2 +-
libminifi/test/unit/NetUtilsTest.cpp | 58 ++++----
minifi-api/include/minifi-cpp/agent/agent_docs.h | 2 +
.../minifi-cpp/controllers/RecordSetReader.h | 6 +
.../minifi-cpp/controllers/RecordSetWriter.h | 7 +
...ntextService.h => SSLContextServiceInterface.h} | 10 +-
...finition.h => ControllerServiceApiDefinition.h} | 9 +-
83 files changed, 730 insertions(+), 325 deletions(-)
copy core-framework/include/Exception.h => extension-framework/include/controllers/SSLContextServiceInterface.h (92%)
rename minifi-api/include/minifi-cpp/controllers/{SSLContextService.h => SSLContextServiceInterface.h} (83%)
copy minifi-api/include/minifi-cpp/core/{RelationshipDefinition.h => ControllerServiceApiDefinition.h} (82%)
(nifi-minifi-cpp) 30/37: Win fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit a1a4cbd87db615f277bff91b9849018c5315c3e6
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Jul 10 16:07:28 2025 +0200
Win fix
---
extensions/systemd/ConsumeJournald.cpp | 2 +-
extensions/windows-event-log/ConsumeWindowsEventLog.cpp | 2 +-
.../windows-event-log/tests/ConsumeWindowsEventLogTests.cpp | 9 +++++++--
3 files changed, 9 insertions(+), 4 deletions(-)
diff --git a/extensions/systemd/ConsumeJournald.cpp b/extensions/systemd/ConsumeJournald.cpp
index 3afbbd899..9bc804c97 100644
--- a/extensions/systemd/ConsumeJournald.cpp
+++ b/extensions/systemd/ConsumeJournald.cpp
@@ -34,7 +34,7 @@ namespace org::apache::nifi::minifi::extensions::systemd {
namespace chr = std::chrono;
ConsumeJournald::ConsumeJournald(core::ProcessorMetadata metadata, std::unique_ptr<libwrapper::LibWrapper>&& libwrapper)
- : core::ProcessorImpl{metadata}, libwrapper_{std::move(libwrapper)} {}
+ : core::ProcessorImpl{std::move(metadata)}, libwrapper_{std::move(libwrapper)} {}
void ConsumeJournald::initialize() {
setSupportedProperties(Properties);
diff --git a/extensions/windows-event-log/ConsumeWindowsEventLog.cpp b/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
index d2bbbf6ea..24ed13c25 100644
--- a/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
+++ b/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
@@ -62,7 +62,7 @@ namespace org::apache::nifi::minifi::processors {
const int EVT_NEXT_TIMEOUT_MS = 500;
ConsumeWindowsEventLog::ConsumeWindowsEventLog(core::ProcessorMetadata metadata)
- : core::ProcessorImpl(metadata) {
+ : core::ProcessorImpl{std::move(metadata)} {
char buff[MAX_COMPUTERNAME_LENGTH + 1];
DWORD size = sizeof(buff);
if (GetComputerName(buff, &size)) {
diff --git a/extensions/windows-event-log/tests/ConsumeWindowsEventLogTests.cpp b/extensions/windows-event-log/tests/ConsumeWindowsEventLogTests.cpp
index f3d0954ed..f8b66e3ae 100644
--- a/extensions/windows-event-log/tests/ConsumeWindowsEventLogTests.cpp
+++ b/extensions/windows-event-log/tests/ConsumeWindowsEventLogTests.cpp
@@ -93,8 +93,13 @@ TEST_CASE("ConsumeWindowsEventLog constructor works", "[create]") {
TestController test_controller;
std::shared_ptr<TestPlan> test_plan = test_controller.createPlan();
- REQUIRE_NOTHROW(ConsumeWindowsEventLog("one"));
- REQUIRE_NOTHROW(ConsumeWindowsEventLog("two", IdGenerator::getIdGenerator()->generate()));
+ core::ProcessorMetadata metadata{
+ .uuid = IdGenerator::getIdGenerator()->generate(),
+ .name = "CWEL",
+ .logger = core::logging::LoggerFactory<ConsumeWindowsEventLog>::getLogger()
+ };
+
+ REQUIRE_NOTHROW(ConsumeWindowsEventLog(metadata));
REQUIRE_NOTHROW(test_plan->addProcessor("ConsumeWindowsEventLog", "cwel"));
}
(nifi-minifi-cpp) 08/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit c04a6d89acba6c03ab87648466ee6aecf973f95b
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Apr 24 11:48:55 2025 +0200
Rebase fix
---
extension-utils/include/core/AbstractProcessor.h | 5 +----
extensions/kubernetes/processors/CollectKubernetesPodMetrics.h | 7 ++-----
extensions/pdh/PerformanceDataMonitor.h | 6 +++---
extensions/procfs/processors/ProcFsMonitor.h | 5 +----
extensions/python/ExecutePythonProcessor.h | 6 +-----
extensions/smb/FetchSmb.h | 5 +----
extensions/smb/ListSmb.h | 5 +----
extensions/smb/PutSmb.h | 5 +----
extensions/windows-event-log/TailEventLog.h | 5 +----
9 files changed, 12 insertions(+), 37 deletions(-)
diff --git a/extension-utils/include/core/AbstractProcessor.h b/extension-utils/include/core/AbstractProcessor.h
index 40710d205..f4a76f57e 100644
--- a/extension-utils/include/core/AbstractProcessor.h
+++ b/extension-utils/include/core/AbstractProcessor.h
@@ -32,10 +32,7 @@ namespace org::apache::nifi::minifi::core {
template<typename ProcessorT>
class AbstractProcessor : public ProcessorImpl {
public:
- explicit AbstractProcessor(std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ProcessorT>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
void initialize() final {
static_assert(std::is_same_v<typename decltype(ProcessorT::Properties)::value_type, PropertyReference>);
diff --git a/extensions/kubernetes/processors/CollectKubernetesPodMetrics.h b/extensions/kubernetes/processors/CollectKubernetesPodMetrics.h
index 31a70cc0d..fae416df8 100644
--- a/extensions/kubernetes/processors/CollectKubernetesPodMetrics.h
+++ b/extensions/kubernetes/processors/CollectKubernetesPodMetrics.h
@@ -21,7 +21,7 @@
#include "../controllerservice/KubernetesControllerService.h"
#include "core/logging/LoggerFactory.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -29,10 +29,7 @@ namespace org::apache::nifi::minifi::processors {
class CollectKubernetesPodMetrics : public core::ProcessorImpl {
public:
- explicit CollectKubernetesPodMetrics(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<CollectKubernetesPodMetrics>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "A processor which collects pod metrics when MiNiFi is run inside Kubernetes.";
diff --git a/extensions/pdh/PerformanceDataMonitor.h b/extensions/pdh/PerformanceDataMonitor.h
index b644d81a5..4e52da822 100644
--- a/extensions/pdh/PerformanceDataMonitor.h
+++ b/extensions/pdh/PerformanceDataMonitor.h
@@ -49,9 +49,9 @@ class PerformanceDataMonitor final : public core::ProcessorImpl {
static constexpr const char* PRETTY_FORMAT_STR = "Pretty";
static constexpr const char* COMPACT_FORMAT_STR = "Compact";
- explicit PerformanceDataMonitor(const std::string_view name, utils::Identifier uuid = utils::Identifier())
- : ProcessorImpl(name, uuid), output_format_(OutputFormat::JSON), pretty_output_(false),
- decimal_places_(std::nullopt), logger_(core::logging::LoggerFactory<PerformanceDataMonitor>::getLogger()),
+ explicit PerformanceDataMonitor(core::ProcessorMetadata info)
+ : ProcessorImpl(info), output_format_(OutputFormat::JSON), pretty_output_(false),
+ decimal_places_(std::nullopt),
pdh_query_(nullptr), resource_consumption_counters_() {}
~PerformanceDataMonitor() override;
diff --git a/extensions/procfs/processors/ProcFsMonitor.h b/extensions/procfs/processors/ProcFsMonitor.h
index c4ce0cd28..d20649771 100644
--- a/extensions/procfs/processors/ProcFsMonitor.h
+++ b/extensions/procfs/processors/ProcFsMonitor.h
@@ -55,10 +55,7 @@ enum class ResultRelativeness {
class ProcFsMonitor final : public core::ProcessorImpl {
public:
- explicit ProcFsMonitor(const std::string_view name, utils::Identifier uuid = utils::Identifier())
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ProcFsMonitor>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~ProcFsMonitor() override = default;
EXTENSIONAPI static constexpr const char* Description = "This processor can create FlowFiles with various performance data through the proc pseudo-filesystem. (Linux only)";
diff --git a/extensions/python/ExecutePythonProcessor.h b/extensions/python/ExecutePythonProcessor.h
index f74ce8cc1..c54f1840f 100644
--- a/extensions/python/ExecutePythonProcessor.h
+++ b/extensions/python/ExecutePythonProcessor.h
@@ -44,10 +44,7 @@ class ExecutePythonProcessor : public core::ProcessorImpl {
: ProcessorImpl(info),
processor_initialized_(false),
python_dynamic_(false),
- reload_on_script_change_(true) {
- logger_ = core::logging::LoggerFactory<ExecutePythonProcessor>::getLogger(uuid_);
- python_logger_ = core::logging::LoggerFactory<ExecutePythonProcessor>::getAliasedLogger(getName());
- }
+ reload_on_script_change_(true) {}
EXTENSIONAPI static constexpr const char* Description = "Executes a script given the flow file and a process session. "
"The script is responsible for handling the incoming flow file (transfer to SUCCESS or remove, e.g.) as well as "
@@ -160,7 +157,6 @@ class ExecutePythonProcessor : public core::ProcessorImpl {
bool reload_on_script_change_;
std::optional<std::chrono::file_clock::time_point> last_script_write_time_;
std::string script_file_path_;
- std::shared_ptr<core::logging::Logger> python_logger_;
std::unique_ptr<PythonScriptEngine> python_script_engine_;
std::optional<std::string> python_class_name_;
std::vector<std::filesystem::path> python_paths_;
diff --git a/extensions/smb/FetchSmb.h b/extensions/smb/FetchSmb.h
index 3d3995b6d..044c51c0c 100644
--- a/extensions/smb/FetchSmb.h
+++ b/extensions/smb/FetchSmb.h
@@ -39,10 +39,7 @@ namespace org::apache::nifi::minifi::extensions::smb {
class FetchSmb final : public core::ProcessorImpl {
public:
- explicit FetchSmb(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<FetchSmb>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Fetches files from a SMB Share. Designed to be used in tandem with ListSmb.";
diff --git a/extensions/smb/ListSmb.h b/extensions/smb/ListSmb.h
index b00e64b04..52cc3566c 100644
--- a/extensions/smb/ListSmb.h
+++ b/extensions/smb/ListSmb.h
@@ -39,10 +39,7 @@ namespace org::apache::nifi::minifi::extensions::smb {
class ListSmb : public core::ProcessorImpl {
public:
- explicit ListSmb(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ListSmb>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Retrieves a listing of files from an SMB share. For each file that is listed, "
"creates a FlowFile that represents the file so that it can be fetched in conjunction with FetchSmb.";
diff --git a/extensions/smb/PutSmb.h b/extensions/smb/PutSmb.h
index a497e3b3c..37ff29242 100644
--- a/extensions/smb/PutSmb.h
+++ b/extensions/smb/PutSmb.h
@@ -31,10 +31,7 @@ namespace org::apache::nifi::minifi::extensions::smb {
class PutSmb final : public core::ProcessorImpl {
public:
- explicit PutSmb(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<PutSmb>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
enum class FileExistsResolutionStrategy {
fail,
diff --git a/extensions/windows-event-log/TailEventLog.h b/extensions/windows-event-log/TailEventLog.h
index e8b606f64..c5bd991b2 100644
--- a/extensions/windows-event-log/TailEventLog.h
+++ b/extensions/windows-event-log/TailEventLog.h
@@ -44,9 +44,7 @@ const char log_name[255] = "Application";
class TailEventLog : public core::ProcessorImpl {
public:
- explicit TailEventLog(const std::string& name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- }
+ using ProcessorImpl::ProcessorImpl;
virtual ~TailEventLog() = default;
EXTENSIONAPI static constexpr const char* Description = "Windows event log reader that functions as a stateful tail of the provided windows event log name";
@@ -129,7 +127,6 @@ class TailEventLog : public core::ProcessorImpl {
DWORD num_records_;
HANDLE log_handle_;
- std::shared_ptr<core::logging::Logger> logger_ = core::logging::LoggerFactory<TailEventLog>::getLogger();
};
} /* namespace processors */
(nifi-minifi-cpp) 10/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit ff80ed560ca972e4f930ff1ccac34c5ceb57053d
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Apr 28 11:37:14 2025 +0200
Rebase fix
---
extensions/gcp/tests/DeleteGCSObjectTests.cpp | 2 +-
extensions/gcp/tests/FetchGCSObjectTests.cpp | 2 +-
extensions/gcp/tests/ListGCSBucketTests.cpp | 2 +-
extensions/gcp/tests/PutGCSObjectTests.cpp | 2 +-
libminifi/test/libtest/unit/ProcessorUtils.h | 46 +++++++++++++++++++++++++++
libminifi/test/libtest/unit/TestUtils.h | 22 +------------
6 files changed, 51 insertions(+), 25 deletions(-)
diff --git a/extensions/gcp/tests/DeleteGCSObjectTests.cpp b/extensions/gcp/tests/DeleteGCSObjectTests.cpp
index f4498d730..bde7e010f 100644
--- a/extensions/gcp/tests/DeleteGCSObjectTests.cpp
+++ b/extensions/gcp/tests/DeleteGCSObjectTests.cpp
@@ -22,7 +22,7 @@
#include "google/cloud/storage/testing/mock_client.h"
#include "google/cloud/storage/internal/object_metadata_parser.h"
#include "google/cloud/storage/testing/canonical_errors.h"
-#include "unit/TestUtils.h"
+#include "unit/ProcessorUtils.h"
namespace gcs = ::google::cloud::storage;
namespace minifi_gcp = org::apache::nifi::minifi::extensions::gcp;
diff --git a/extensions/gcp/tests/FetchGCSObjectTests.cpp b/extensions/gcp/tests/FetchGCSObjectTests.cpp
index c233f8d67..ea87c7e63 100644
--- a/extensions/gcp/tests/FetchGCSObjectTests.cpp
+++ b/extensions/gcp/tests/FetchGCSObjectTests.cpp
@@ -22,7 +22,7 @@
#include "google/cloud/storage/testing/mock_client.h"
#include "google/cloud/storage/internal/object_metadata_parser.h"
#include "google/cloud/storage/testing/canonical_errors.h"
-#include "unit/TestUtils.h"
+#include "unit/ProcessorUtils.h"
namespace gcs = ::google::cloud::storage;
namespace minifi_gcp = org::apache::nifi::minifi::extensions::gcp;
diff --git a/extensions/gcp/tests/ListGCSBucketTests.cpp b/extensions/gcp/tests/ListGCSBucketTests.cpp
index e0b435a13..16b2c8dad 100644
--- a/extensions/gcp/tests/ListGCSBucketTests.cpp
+++ b/extensions/gcp/tests/ListGCSBucketTests.cpp
@@ -21,7 +21,7 @@
#include "google/cloud/storage/testing/mock_client.h"
#include "google/cloud/storage/internal/object_metadata_parser.h"
#include "google/cloud/storage/testing/canonical_errors.h"
-#include "unit/TestUtils.h"
+#include "unit/ProcessorUtils.h"
namespace gcs = ::google::cloud::storage;
namespace minifi_gcp = org::apache::nifi::minifi::extensions::gcp;
diff --git a/extensions/gcp/tests/PutGCSObjectTests.cpp b/extensions/gcp/tests/PutGCSObjectTests.cpp
index f099b3d77..ee7c26ca3 100644
--- a/extensions/gcp/tests/PutGCSObjectTests.cpp
+++ b/extensions/gcp/tests/PutGCSObjectTests.cpp
@@ -24,7 +24,7 @@
#include "google/cloud/storage/internal/object_metadata_parser.h"
#include "google/cloud/storage/retry_policy.h"
#include "google/cloud/storage/testing/canonical_errors.h"
-#include "unit/TestUtils.h"
+#include "unit/ProcessorUtils.h"
namespace gcs = ::google::cloud::storage;
namespace minifi_gcp = org::apache::nifi::minifi::extensions::gcp;
diff --git a/libminifi/test/libtest/unit/ProcessorUtils.h b/libminifi/test/libtest/unit/ProcessorUtils.h
new file mode 100644
index 000000000..36a270d9d
--- /dev/null
+++ b/libminifi/test/libtest/unit/ProcessorUtils.h
@@ -0,0 +1,46 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include "core/Processor.h"
+#include "minifi-cpp/core/ProcessorMetadata.h"
+
+namespace org::apache::nifi::minifi::test::utils {
+
+template<typename T>
+std::unique_ptr<core::Processor> make_processor(std::string_view name, std::optional<minifi::utils::Identifier> uuid = std::nullopt) {
+ if (!uuid) {
+ uuid = minifi::utils::IdGenerator::getIdGenerator()->generate();
+ }
+ auto processor_impl = std::make_unique<T>(core::ProcessorMetadata{
+ .uuid = uuid.value(),
+ .name = std::string{name},
+ .logger = minifi::core::logging::LoggerFactory<T>::getLogger(uuid.value())
+ });
+ return std::make_unique<core::Processor>(name, uuid.value(), std::move(processor_impl));
+}
+
+template<typename T, typename ...Args>
+std::unique_ptr<core::Processor> make_custom_processor(Args&&... args) {
+ auto processor_impl = std::make_unique<T>(std::forward<Args>(args)...);
+ auto name = processor_impl->getName();
+ auto uuid = processor_impl->getUUID();
+ return std::make_unique<core::Processor>(name, uuid, std::move(processor_impl));
+}
+
+} // namespace org::apache::nifi::minifi::test::utils
\ No newline at end of file
diff --git a/libminifi/test/libtest/unit/TestUtils.h b/libminifi/test/libtest/unit/TestUtils.h
index ff3b435a3..83e9bce23 100644
--- a/libminifi/test/libtest/unit/TestUtils.h
+++ b/libminifi/test/libtest/unit/TestUtils.h
@@ -41,6 +41,7 @@
#include "range/v3/algorithm/any_of.hpp"
#include "core/Processor.h"
#include "core/logging/LoggerFactory.h"
+#include "./ProcessorUtils.h"
using namespace std::literals::chrono_literals;
@@ -233,27 +234,6 @@ inline bool runningAsUnixRoot() {
#endif
}
-template<typename T>
-std::unique_ptr<core::Processor> make_processor(std::string_view name, std::optional<minifi::utils::Identifier> uuid = std::nullopt) {
- if (!uuid) {
- uuid = minifi::utils::IdGenerator::getIdGenerator()->generate();
- }
- auto processor_impl = std::make_unique<T>(core::ProcessorMetadata{
- .uuid = uuid.value(),
- .name = std::string{name},
- .logger = minifi::core::logging::LoggerFactory<T>::getLogger(uuid.value())
- });
- return std::make_unique<core::Processor>(name, uuid.value(), std::move(processor_impl));
-}
-
-template<typename T, typename ...Args>
-std::unique_ptr<core::Processor> make_custom_processor(Args&&... args) {
- auto processor_impl = std::make_unique<T>(std::forward<Args>(args)...);
- auto name = processor_impl->getName();
- auto uuid = processor_impl->getUUID();
- return std::make_unique<core::Processor>(name, uuid, std::move(processor_impl));
-}
-
} // namespace org::apache::nifi::minifi::test::utils
namespace Catch {
(nifi-minifi-cpp) 01/37: Working on api reduction
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 5be79f51e5508fba37242ef534fedd1c5620b607
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Tue Feb 18 12:37:56 2025 +0100
Working on api reduction
---
.../include/utils/ProcessorConfigUtils.h | 1 +
extension-utils/src/utils/ProcessorConfigUtils.cpp | 108 +++++++
extension-utils/src/utils/net/Ssl.cpp | 2 +-
extensions/civetweb/processors/ListenHTTP.cpp | 10 +-
extensions/civetweb/processors/ListenHTTP.h | 2 -
.../controllerservices/CouchbaseClusterService.cpp | 2 +-
.../expression-language/ProcessContextExpr.cpp | 32 +-
.../expression-language/ProcessContextExpr.h | 28 +-
extensions/grafana-loki/PushGrafanaLoki.cpp | 8 -
extensions/grafana-loki/PushGrafanaLoki.h | 1 -
extensions/libarchive/BinFiles.cpp | 8 -
extensions/libarchive/BinFiles.h | 2 -
extensions/python/ExecutePythonProcessor.cpp | 76 +----
extensions/python/ExecutePythonProcessor.h | 2 +-
extensions/python/types/PyProcessContext.cpp | 6 +-
extensions/smb/SmbConnectionControllerService.cpp | 2 +-
.../processors/AttributeRollingWindow.cpp | 2 +-
.../processors/DefragmentText.cpp | 8 -
.../processors/DefragmentText.h | 1 -
.../standard-processors/processors/ExtractText.h | 5 +-
.../processors/GenerateFlowFile.cpp | 5 +-
.../processors/GenerateFlowFile.h | 5 +-
.../standard-processors/processors/GetFile.h | 2 +-
.../processors/RouteOnAttribute.cpp | 6 +-
.../processors/RouteOnAttribute.h | 5 +-
.../processors/SegmentContent.h | 4 +-
.../standard-processors/processors/SplitText.h | 5 +-
.../processors/UpdateAttribute.h | 5 +-
libminifi/include/ForwardingNode.h | 7 +-
libminifi/include/RemoteProcessorGroupPort.h | 4 +-
.../include}/core/Processor.h | 26 +-
.../include/core/ProcessorProxy.h | 110 ++-----
.../include/core/state/nodes/FlowInformation.h | 2 +-
libminifi/include/core/state/nodes/MetricsBase.h | 2 +-
libminifi/src/RemoteProcessorGroupPort.cpp | 14 +-
libminifi/src/core/ClassLoader.cpp | 44 +++
libminifi/src/core/FlowConfiguration.cpp | 1 +
libminifi/src/core/ProcessContext.cpp | 110 +++++++
libminifi/src/core/ProcessSession.cpp | 1 +
{utils => libminifi}/src/core/Processor.cpp | 95 +++---
libminifi/test/libtest/unit/DummyProcessor.h | 2 +-
.../libtest/unit/ReadFromFlowFileTestProcessor.h | 2 +-
libminifi/test/libtest/unit/StatefulProcessor.h | 2 +-
libminifi/test/libtest/unit/TestBase.cpp | 2 +-
minifi-api/include/minifi-cpp/core/ClassLoader.h | 3 +
.../include/minifi-cpp/core/ProcessContext.h | 19 ++
.../minifi-cpp/core/ProcessContextBuilder.h | 2 +
minifi-api/include/minifi-cpp/core/Processor.h | 57 +---
.../include/minifi-cpp/core/ProcessorDescriptor.h | 33 ++
.../include/minifi-cpp/core/ProcessorFactory.h | 37 +++
.../include/minifi-cpp/core/ProcessorMetadata.h | 32 ++
.../minifi-cpp/core/state/nodes/MetricsBase.h | 4 +-
utils/include/core/ProcessContext.h | 65 +---
utils/include/core/Processor.h | 288 +-----------------
utils/include/core/ProcessorFactory.h | 57 ++++
utils/include/core/ProcessorImpl.h | 159 ++++++++++
utils/include/core/ProcessorMetrics.h | 6 +-
utils/include/core/Resource.h | 14 +-
utils/include/core/state/nodes/ResponseNode.h | 20 +-
utils/src/core/Processor.cpp | 337 +++------------------
utils/src/core/ProcessorMetrics.cpp | 2 +-
61 files changed, 855 insertions(+), 1047 deletions(-)
diff --git a/extension-utils/include/utils/ProcessorConfigUtils.h b/extension-utils/include/utils/ProcessorConfigUtils.h
index dbb99f748..29de57eec 100644
--- a/extension-utils/include/utils/ProcessorConfigUtils.h
+++ b/extension-utils/include/utils/ProcessorConfigUtils.h
@@ -27,6 +27,7 @@
#include "utils/Enum.h"
#include "utils/expected.h"
#include "utils/OptionalUtils.h"
+#include "minifi-cpp/Exception.h"
namespace org::apache::nifi::minifi::utils {
diff --git a/extension-utils/src/utils/ProcessorConfigUtils.cpp b/extension-utils/src/utils/ProcessorConfigUtils.cpp
new file mode 100644
index 000000000..2ba8c73cb
--- /dev/null
+++ b/extension-utils/src/utils/ProcessorConfigUtils.cpp
@@ -0,0 +1,108 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "utils/ProcessorConfigUtils.h"
+
+#include <string>
+
+#include "utils/expected.h"
+
+namespace org::apache::nifi::minifi::utils {
+
+std::string parseProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ return ctx.getProperty(property, flow_file) | utils::expect(fmt::format("Expected valid value from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+}
+
+std::optional<std::string> parseOptionalProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ if (const auto property_str = ctx.getProperty(property, flow_file)) {
+ return *property_str;
+ }
+ return std::nullopt;
+}
+
+std::optional<bool> parseOptionalBoolProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ if (const auto property_str = ctx.getProperty(property, flow_file)) {
+ return parsing::parseBool(*property_str) | utils::expect(fmt::format("Expected parsable bool from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+ }
+ return std::nullopt;
+}
+
+bool parseBoolProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseBool) | utils::expect(fmt::format("Expected parsable bool from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+}
+
+std::optional<uint64_t> parseOptionalU64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ if (const auto property_str = ctx.getProperty(property, flow_file)) {
+ if (property_str->empty()) {
+ return std::nullopt;
+ }
+ return parsing::parseIntegral<uint64_t>(*property_str) | utils::expect(fmt::format("Expected parsable uint64_t from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+ }
+
+ return std::nullopt;
+}
+
+uint64_t parseU64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseIntegral<uint64_t>) | utils::expect(fmt::format("Expected parsable uint64_t from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+}
+
+std::optional<int64_t> parseOptionalI64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ if (const auto property_str = ctx.getProperty(property, flow_file)) {
+ if (property_str->empty()) {
+ return std::nullopt;
+ }
+ return parsing::parseIntegral<int64_t>(*property_str) | utils::expect(fmt::format("Expected parsable int64_t from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+ }
+
+ return std::nullopt;
+}
+
+int64_t parseI64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseIntegral<int64_t>) | utils::expect(fmt::format("Expected parsable int64_t from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+}
+
+std::optional<std::chrono::milliseconds> parseOptionalMsProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ if (const auto property_str = ctx.getProperty(property, flow_file)) {
+ if (property_str->empty()) {
+ return std::nullopt;
+ }
+ return parsing::parseDuration(*property_str) | utils::expect(fmt::format("Expected parsable duration from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+ }
+
+ return std::nullopt;
+}
+
+std::chrono::milliseconds parseMsProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseDuration<std::chrono::milliseconds>) | utils::expect(fmt::format("Expected parsable duration from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+}
+
+std::optional<uint64_t> parseOptionalDataSizeProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ if (const auto property_str = ctx.getProperty(property, flow_file)) {
+ if (property_str->empty()) {
+ return std::nullopt;
+ }
+ return parsing::parseDataSize(*property_str) | utils::expect(fmt::format("Expected parsable data size from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+ }
+
+ return std::nullopt;
+}
+
+uint64_t parseDataSizeProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
+ return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseDataSize) | utils::expect(fmt::format("Expected parsable data size from {}::{}", ctx.getProcessorInfo().getName(), property.name));
+}
+
+
+} // namespace org::apache::nifi::minifi::utils
diff --git a/extension-utils/src/utils/net/Ssl.cpp b/extension-utils/src/utils/net/Ssl.cpp
index 0f42fa6b0..b40f87a55 100644
--- a/extension-utils/src/utils/net/Ssl.cpp
+++ b/extension-utils/src/utils/net/Ssl.cpp
@@ -22,7 +22,7 @@ namespace org::apache::nifi::minifi::utils::net {
std::optional<utils::net::SslData> getSslData(const core::ProcessContext& context, const core::PropertyReference& ssl_prop, const std::shared_ptr<core::logging::Logger>& logger) {
auto getSslContextService = [&]() -> std::shared_ptr<minifi::controllers::SSLContextServiceInterface> {
if (auto ssl_service_name = context.getProperty(ssl_prop); ssl_service_name && !ssl_service_name->empty()) {
- if (auto service = context.getControllerService(*ssl_service_name, context.getProcessor().getUUID())) {
+ if (auto service = context.getControllerService(*ssl_service_name, context.getProcessorInfo().getUUID())) {
if (auto ssl_service = std::dynamic_pointer_cast<org::apache::nifi::minifi::controllers::SSLContextServiceInterface>(service)) {
return ssl_service;
} else {
diff --git a/extensions/civetweb/processors/ListenHTTP.cpp b/extensions/civetweb/processors/ListenHTTP.cpp
index d2096643d..46f627f34 100644
--- a/extensions/civetweb/processors/ListenHTTP.cpp
+++ b/extensions/civetweb/processors/ListenHTTP.cpp
@@ -108,7 +108,7 @@ void ListenHTTP::onSchedule(core::ProcessContext& context, core::ProcessSessionF
logger_->log_debug("ListenHTTP using {}: {}", BatchSize.name, batch_size_);
std::optional<std::string> flow_id;
- if (auto flow_version = context.getProcessor().getFlowIdentifier()) {
+ if (auto flow_version = context.getProcessorInfo().getFlowIdentifier()) {
flow_id = flow_version->getFlowId();
}
@@ -504,14 +504,6 @@ void ListenHTTP::notifyStop() {
handler_.reset();
}
-std::set<core::Connectable*> ListenHTTP::getOutGoingConnections(const std::string &relationship) {
- auto result = core::ProcessorImpl::getOutGoingConnections(relationship);
- if (relationship == Self.name) {
- result.insert(this);
- }
- return result;
-}
-
REGISTER_RESOURCE(ListenHTTP, Processor);
} // namespace org::apache::nifi::minifi::processors
diff --git a/extensions/civetweb/processors/ListenHTTP.h b/extensions/civetweb/processors/ListenHTTP.h
index ce47ab97c..4a5181988 100644
--- a/extensions/civetweb/processors/ListenHTTP.h
+++ b/extensions/civetweb/processors/ListenHTTP.h
@@ -155,8 +155,6 @@ class ListenHTTP : public core::ProcessorImpl {
return handler_ ? !handler_->empty() : false;
}
- std::set<core::Connectable*> getOutGoingConnections(const std::string &relationship) override;
-
struct ResponseBody {
std::string uri;
std::string mime_type;
diff --git a/extensions/couchbase/controllerservices/CouchbaseClusterService.cpp b/extensions/couchbase/controllerservices/CouchbaseClusterService.cpp
index d8dc2066f..f85203e6d 100644
--- a/extensions/couchbase/controllerservices/CouchbaseClusterService.cpp
+++ b/extensions/couchbase/controllerservices/CouchbaseClusterService.cpp
@@ -250,7 +250,7 @@ void CouchbaseClusterService::onEnable() {
gsl::not_null<std::shared_ptr<CouchbaseClusterService>> CouchbaseClusterService::getFromProperty(const core::ProcessContext& context, const core::PropertyReference& property) {
std::shared_ptr<CouchbaseClusterService> couchbase_cluster_service;
if (auto connection_controller_name = context.getProperty(property)) {
- couchbase_cluster_service = std::dynamic_pointer_cast<CouchbaseClusterService>(context.getControllerService(*connection_controller_name, context.getProcessor().getUUID()));
+ couchbase_cluster_service = std::dynamic_pointer_cast<CouchbaseClusterService>(context.getControllerService(*connection_controller_name, context.getProcessorInfo().getUUID()));
}
if (!couchbase_cluster_service) {
throw minifi::Exception(ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "Missing Couchbase Cluster Service");
diff --git a/extensions/expression-language/ProcessContextExpr.cpp b/extensions/expression-language/ProcessContextExpr.cpp
index d22447c5d..bff17c6b6 100644
--- a/extensions/expression-language/ProcessContextExpr.cpp
+++ b/extensions/expression-language/ProcessContextExpr.cpp
@@ -25,21 +25,21 @@
namespace org::apache::nifi::minifi::core {
-nonstd::expected<std::string, std::error_code> ProcessContextExpr::getProperty(const std::string_view name, const FlowFile* flow_file) const {
- const auto property = getProcessor().getSupportedProperty(name);
+nonstd::expected<std::string, std::error_code> ProcessContextExpr::getProperty(ProcessContext& context, const std::string_view name, const FlowFile* flow_file) const {
+ const auto property = context.getProcessorInfo().getPropertyReference(name);
if (!property) {
return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty);
}
- if (!property->supportsExpressionLanguage()) {
- return ProcessContextImpl::getProperty(name, flow_file);
+ if (!property->supports_expression_language) {
+ return context.getProperty(name, flow_file);
}
if (!cached_expressions_.contains(name)) {
- auto expression_str = ProcessContextImpl::getProperty(name, flow_file);
+ auto expression_str = context.getProperty(name, flow_file);
if (!expression_str) { return expression_str; }
cached_expressions_.emplace(std::string{name}, expression::compile(*expression_str));
}
- expression::Parameters p(this, flow_file);
+ expression::Parameters p(&context, flow_file);
auto result = cached_expressions_[std::string{name}](p).asString();
if (!property->getValidator().validate(result)) {
return nonstd::make_unexpected(PropertyErrorCode::ValidationFailed);
@@ -47,21 +47,25 @@ nonstd::expected<std::string, std::error_code> ProcessContextExpr::getProperty(c
return result;
}
-nonstd::expected<std::string, std::error_code> ProcessContextExpr::getDynamicProperty(const std::string_view name, const FlowFile* flow_file) const {
- // all dynamic properties support EL
+nonstd::expected<std::string, std::error_code> ProcessContextExpr::getDynamicProperty(ProcessContext& context, const std::string_view name, const FlowFile* flow_file) const {
if (!cached_dynamic_expressions_.contains(name)) {
- auto expression_str = ProcessContextImpl::getDynamicProperty(name, flow_file);
+ auto expression_str = context.getDynamicProperty(name, flow_file);
if (!expression_str) { return expression_str; }
cached_dynamic_expressions_.emplace(std::string{name}, expression::compile(*expression_str));
}
- const expression::Parameters p(this, flow_file);
+ const expression::Parameters p(&context, flow_file);
return cached_dynamic_expressions_[std::string{name}](p).asString();
}
-nonstd::expected<void, std::error_code> ProcessContextExpr::setProperty(const std::string_view name, std::string value) {
- cached_expressions_.erase(std::string{name});
- return ProcessContextImpl::setProperty(name, std::move(value));
-}
+//nonstd::expected<void, std::error_code> ProcessContextExpr::setProperty(const std::string_view name, std::string value) {
+// cached_expressions_.erase(std::string{name});
+// return ProcessContextImpl::setProperty(name, std::move(value));
+//}
+//
+//nonstd::expected<void, std::error_code> ProcessContextExpr::setDynamicProperty(std::string name, std::string value) {
+// cached_dynamic_expressions_.erase(name);
+// return ProcessContextImpl::setDynamicProperty(std::move(name), std::move(value));
+//}
nonstd::expected<void, std::error_code> ProcessContextExpr::setDynamicProperty(std::string name, std::string value) {
cached_dynamic_expressions_.erase(name);
diff --git a/extensions/expression-language/ProcessContextExpr.h b/extensions/expression-language/ProcessContextExpr.h
index fdd00caef..cb33c973c 100644
--- a/extensions/expression-language/ProcessContextExpr.h
+++ b/extensions/expression-language/ProcessContextExpr.h
@@ -34,35 +34,19 @@ namespace org::apache::nifi::minifi::core {
*/
class ProcessContextExpr final : public core::ProcessContextImpl {
public:
- ProcessContextExpr(Processor& processor, controller::ControllerServiceProvider* controller_service_provider,
- const std::shared_ptr<core::Repository> &repo, const std::shared_ptr<core::Repository> &flow_repo,
- const std::shared_ptr<core::ContentRepository> &content_repo = core::repository::createFileSystemRepository())
- : core::ProcessContextImpl(processor, controller_service_provider, repo, flow_repo, content_repo),
- logger_(logging::LoggerFactory<ProcessContextExpr>::getLogger()) {
- }
-
- ProcessContextExpr(Processor& processor, controller::ControllerServiceProvider* controller_service_provider,
- const std::shared_ptr<core::Repository> &repo, const std::shared_ptr<core::Repository> &flow_repo, const std::shared_ptr<minifi::Configure> &configuration,
- const std::shared_ptr<core::ContentRepository> &content_repo = core::repository::createFileSystemRepository())
- : core::ProcessContextImpl(processor, controller_service_provider, repo, flow_repo, configuration, content_repo),
- logger_(logging::LoggerFactory<ProcessContextExpr>::getLogger()) {
- }
-
~ProcessContextExpr() override = default;
- nonstd::expected<std::string, std::error_code> getProperty(std::string_view name, const FlowFile*) const override;
- nonstd::expected<std::string, std::error_code> getDynamicProperty(std::string_view name, const FlowFile*) const override;
+ nonstd::expected<std::string, std::error_code> getProperty(ProcessContext& context, std::string_view name, const FlowFile*) const override;
+ nonstd::expected<std::string, std::error_code> getDynamicProperty(ProcessContext& context, std::string_view name, const FlowFile*) const override;
+//
+// nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value);
+// nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value);
- nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value) override;
- nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value) override;
-
- std::map<std::string, std::string> getDynamicProperties(const FlowFile*) const override;
+ std::map<std::string, std::string> getDynamicProperties(ProcessContext& context, const FlowFile*) const override;
private:
mutable std::unordered_map<std::string, expression::Expression, utils::string::transparent_string_hash, std::equal_to<>> cached_expressions_;
mutable std::unordered_map<std::string, expression::Expression, utils::string::transparent_string_hash, std::equal_to<>> cached_dynamic_expressions_;
-
- std::shared_ptr<logging::Logger> logger_;
};
} // namespace org::apache::nifi::minifi::core
diff --git a/extensions/grafana-loki/PushGrafanaLoki.cpp b/extensions/grafana-loki/PushGrafanaLoki.cpp
index 29eba22d5..f328078ac 100644
--- a/extensions/grafana-loki/PushGrafanaLoki.cpp
+++ b/extensions/grafana-loki/PushGrafanaLoki.cpp
@@ -232,12 +232,4 @@ void PushGrafanaLoki::restore(const std::shared_ptr<core::FlowFile>& flow_file)
log_batch_.restore(flow_file);
}
-std::set<core::Connectable*> PushGrafanaLoki::getOutGoingConnections(const std::string &relationship) {
- auto result = core::ConnectableImpl::getOutGoingConnections(relationship);
- if (relationship == Self.getName()) {
- result.insert(this);
- }
- return result;
-}
-
} // namespace org::apache::nifi::minifi::extensions::grafana::loki
diff --git a/extensions/grafana-loki/PushGrafanaLoki.h b/extensions/grafana-loki/PushGrafanaLoki.h
index d33569058..13e8f577f 100644
--- a/extensions/grafana-loki/PushGrafanaLoki.h
+++ b/extensions/grafana-loki/PushGrafanaLoki.h
@@ -107,7 +107,6 @@ class PushGrafanaLoki : public core::ProcessorImpl {
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& sessionFactory) override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
void restore(const std::shared_ptr<core::FlowFile>& flow_file) override;
- std::set<core::Connectable*> getOutGoingConnections(const std::string &relationship) override;
protected:
static const core::Relationship Self;
diff --git a/extensions/libarchive/BinFiles.cpp b/extensions/libarchive/BinFiles.cpp
index be12f3591..b9366d6d6 100644
--- a/extensions/libarchive/BinFiles.cpp
+++ b/extensions/libarchive/BinFiles.cpp
@@ -317,14 +317,6 @@ void BinFiles::restore(const std::shared_ptr<core::FlowFile>& flowFile) {
file_store_.put(flowFile);
}
-std::set<core::Connectable*> BinFiles::getOutGoingConnections(const std::string &relationship) {
- auto result = core::ConnectableImpl::getOutGoingConnections(relationship);
- if (relationship == Self.getName()) {
- result.insert(this);
- }
- return result;
-}
-
REGISTER_RESOURCE(BinFiles, Processor);
} // namespace org::apache::nifi::minifi::processors
diff --git a/extensions/libarchive/BinFiles.h b/extensions/libarchive/BinFiles.h
index f4b5407bb..5d89c1790 100644
--- a/extensions/libarchive/BinFiles.h
+++ b/extensions/libarchive/BinFiles.h
@@ -262,8 +262,6 @@ class BinFiles : public core::ProcessorImpl {
void restore(const std::shared_ptr<core::FlowFile>& flowFile) override;
- std::set<core::Connectable*> getOutGoingConnections(const std::string &relationship) override;
-
protected:
// Allows general pre-processing of a flow file before it is offered to a bin. This is called before getGroupId().
virtual void preprocessFlowFile(const std::shared_ptr<core::FlowFile>& flow);
diff --git a/extensions/python/ExecutePythonProcessor.cpp b/extensions/python/ExecutePythonProcessor.cpp
index 83ad37405..c63544830 100644
--- a/extensions/python/ExecutePythonProcessor.cpp
+++ b/extensions/python/ExecutePythonProcessor.cpp
@@ -31,16 +31,12 @@
#include "utils/StringUtils.h"
#include "utils/file/FileUtils.h"
#include "utils/ProcessorConfigUtils.h"
-#include "utils/PropertyErrors.h"
+#include "minifi-cpp/utils/PropertyErrors.h"
+#include "minifi-cpp/core/ProcessorDescriptor.h"
namespace org::apache::nifi::minifi::extensions::python::processors {
-void ExecutePythonProcessor::initialize() {
- if (getSupportedProperties().empty()) {
- setSupportedProperties(Properties);
- setSupportedRelationships(Relationships);
- }
-
+void ExecutePythonProcessor::initialize(core::ProcessorDescriptor& self) {
if (processor_initialized_) {
logger_->log_debug("Processor has already been initialized, returning...");
return;
@@ -56,6 +52,13 @@ void ExecutePythonProcessor::initialize() {
// so that we can provide manifest of processor identity on C2
python_script_engine_ = createScriptEngine();
initalizeThroughScriptEngine();
+
+ std::vector<core::PropertyReference> all_properties{Properties.begin(), Properties.end()};
+ for (auto& python_prop : python_properties_) {
+ all_properties.push_back(python_prop.getReference());
+ }
+ self.setSupportedProperties(all_properties);
+ self.setSupportedRelationships(Relationships);
}
void ExecutePythonProcessor::initalizeThroughScriptEngine() {
@@ -80,7 +83,7 @@ void ExecutePythonProcessor::initalizeThroughScriptEngine() {
}
void ExecutePythonProcessor::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& /*sessionFactory*/) {
- addAutoTerminatedRelationship(Original);
+ context.addAutoTerminatedRelationship(Original);
if (!processor_initialized_) {
loadScript();
python_script_engine_ = createScriptEngine();
@@ -217,63 +220,8 @@ void ExecutePythonProcessor::addProperty(const std::string &name, const std::str
python_properties_.emplace_back(property);
}
-nonstd::expected<std::string, std::error_code> ExecutePythonProcessor::getProperty(std::string_view name) const {
- if (auto non_python_property = ConfigurableComponentImpl::getProperty(name)) {
- return *non_python_property;
- }
- std::lock_guard<std::mutex> lock(python_properties_mutex_);
- auto it = ranges::find_if(python_properties_, [&name](const auto& item){
- return item.getName() == name;
- });
- if (it != python_properties_.end()) {
- return it->getValue() | utils::transform([](const std::string_view value_view) -> std::string { return std::string{value_view}; });
- }
- return nonstd::make_unexpected(core::PropertyErrorCode::NotSupportedProperty);
-}
-
-nonstd::expected<void, std::error_code> ExecutePythonProcessor::setProperty(std::string_view name, std::string value) {
- auto set_non_python_property = ConfigurableComponentImpl::setProperty(name, value);
- if (set_non_python_property || set_non_python_property.error() != core::PropertyErrorCode::NotSupportedProperty) {
- return set_non_python_property;
- }
- std::lock_guard<std::mutex> lock(python_properties_mutex_);
- auto it = ranges::find_if(python_properties_, [&name](const auto& item){
- return item.getName() == name;
- });
- if (it != python_properties_.end()) {
- return it->setValue(std::move(value));
- }
- return nonstd::make_unexpected(core::PropertyErrorCode::NotSupportedProperty);
-}
-
-nonstd::expected<core::Property, std::error_code> ExecutePythonProcessor::getSupportedProperty(std::string_view name) const {
- if (const auto non_python_property = ConfigurableComponentImpl::getSupportedProperty(name)) {
- return *non_python_property;
- }
- std::lock_guard<std::mutex> lock(python_properties_mutex_);
- auto it = ranges::find_if(python_properties_, [&name](const auto& item){
- return item.getName() == name;
- });
- if (it != python_properties_.end()) {
- return *it;
- }
- return nonstd::make_unexpected(core::PropertyErrorCode::NotSupportedProperty);
-}
-
-std::map<std::string, core::Property, std::less<>> ExecutePythonProcessor::getSupportedProperties() const {
- auto result = ConfigurableComponentImpl::getSupportedProperties();
-
- std::lock_guard<std::mutex> lock(python_properties_mutex_);
-
- for (const auto &property : python_properties_) {
- result.insert({ property.getName(), property });
- }
-
- return result;
-}
-
std::vector<core::Relationship> ExecutePythonProcessor::getPythonRelationships() const {
- auto relationships = getSupportedRelationships();
+ std::vector<core::Relationship> relationships{Relationships.begin(), Relationships.end()};
auto custom_relationships = python_script_engine_->getCustomPythonRelationships();
relationships.reserve(relationships.size() + std::distance(custom_relationships.begin(), custom_relationships.end()));
relationships.insert(relationships.end(), custom_relationships.begin(), custom_relationships.end());
diff --git a/extensions/python/ExecutePythonProcessor.h b/extensions/python/ExecutePythonProcessor.h
index b9e12da07..caefb7b6a 100644
--- a/extensions/python/ExecutePythonProcessor.h
+++ b/extensions/python/ExecutePythonProcessor.h
@@ -90,7 +90,7 @@ class ExecutePythonProcessor : public core::ProcessorImpl {
EXTENSIONAPI static constexpr bool IsSingleThreaded = true;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- void initialize() override;
+ void initialize(core::ProcessorDescriptor& self) override;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
diff --git a/extensions/python/types/PyProcessContext.cpp b/extensions/python/types/PyProcessContext.cpp
index 0d39f6ce7..910e1b3f4 100644
--- a/extensions/python/types/PyProcessContext.cpp
+++ b/extensions/python/types/PyProcessContext.cpp
@@ -123,7 +123,7 @@ PyObject* PyProcessContext::getControllerService(PyProcessContext* self, PyObjec
return nullptr;
}
- if (auto controller_service = context->getControllerService(controller_service_name, context->getProcessor().getUUID())) {
+ if (auto controller_service = context->getControllerService(controller_service_name, context->getProcessorInfo().getUUID())) {
std::string controller_service_type_str = controller_service_type;
if (controller_service_type_str == "SSLContextService") {
auto ssl_ctx_service = std::dynamic_pointer_cast<controllers::SSLContextServiceInterface>(controller_service);
@@ -147,7 +147,7 @@ PyObject* PyProcessContext::getName(PyProcessContext* self, PyObject*) {
return nullptr;
}
- return object::returnReference(context->getProcessor().getName());
+ return object::returnReference(context->getProcessorInfo().getName());
}
PyObject* PyProcessContext::getProperties(PyProcessContext* self, PyObject*) {
@@ -157,7 +157,7 @@ PyObject* PyProcessContext::getProperties(PyProcessContext* self, PyObject*) {
return nullptr;
}
- auto properties = context->getProcessor().getSupportedProperties();
+ auto properties = context->getProcessorInfo().getSupportedProperties();
auto py_properties = OwnedDict::create();
for (const auto& [property_name, property] : properties) {
if (const auto value = context->getProperty(property_name)) {
diff --git a/extensions/smb/SmbConnectionControllerService.cpp b/extensions/smb/SmbConnectionControllerService.cpp
index 87876dd95..d1302b32c 100644
--- a/extensions/smb/SmbConnectionControllerService.cpp
+++ b/extensions/smb/SmbConnectionControllerService.cpp
@@ -61,7 +61,7 @@ void SmbConnectionControllerService::notifyStop() {
gsl::not_null<std::shared_ptr<SmbConnectionControllerService>> SmbConnectionControllerService::getFromProperty(const core::ProcessContext& context, const core::PropertyReference& property) {
std::shared_ptr<SmbConnectionControllerService> smb_connection_controller_service;
if (auto connection_controller_name = context.getProperty(property)) {
- smb_connection_controller_service = std::dynamic_pointer_cast<SmbConnectionControllerService>(context.getControllerService(*connection_controller_name, context.getProcessor().getUUID()));
+ smb_connection_controller_service = std::dynamic_pointer_cast<SmbConnectionControllerService>(context.getControllerService(*connection_controller_name, context.getProcessorInfo().getUUID()));
}
if (!smb_connection_controller_service) {
throw minifi::Exception(ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "Missing SMB Connection Controller Service");
diff --git a/extensions/standard-processors/processors/AttributeRollingWindow.cpp b/extensions/standard-processors/processors/AttributeRollingWindow.cpp
index 922967577..e3108d32c 100644
--- a/extensions/standard-processors/processors/AttributeRollingWindow.cpp
+++ b/extensions/standard-processors/processors/AttributeRollingWindow.cpp
@@ -46,7 +46,7 @@ void AttributeRollingWindow::onSchedule(core::ProcessContext& context, core::Pro
void AttributeRollingWindow::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
gsl_Expects(runningInvariant());
const auto flow_file = session.get();
- if (!flow_file) { yield(); return; }
+ if (!flow_file) { context.yield(); return; }
gsl_Assert(flow_file);
const auto current_value_opt_str = context.getProperty(ValueToTrack, flow_file.get());
if (!current_value_opt_str) {
diff --git a/extensions/standard-processors/processors/DefragmentText.cpp b/extensions/standard-processors/processors/DefragmentText.cpp
index a85a47391..e5c85dc77 100644
--- a/extensions/standard-processors/processors/DefragmentText.cpp
+++ b/extensions/standard-processors/processors/DefragmentText.cpp
@@ -188,14 +188,6 @@ void DefragmentText::restore(const std::shared_ptr<core::FlowFile>& flowFile) {
flow_file_store_.put(flowFile);
}
-std::set<core::Connectable*> DefragmentText::getOutGoingConnections(const std::string &relationship) {
- auto result = core::ConnectableImpl::getOutGoingConnections(relationship);
- if (relationship == Self.getName()) {
- result.insert(this);
- }
- return result;
-}
-
void DefragmentText::Buffer::append(core::ProcessSession& session, const gsl::not_null<std::shared_ptr<core::FlowFile>>& flow_file_to_append) {
if (empty()) {
store(session, flow_file_to_append);
diff --git a/extensions/standard-processors/processors/DefragmentText.h b/extensions/standard-processors/processors/DefragmentText.h
index 0b49962d1..518134afd 100644
--- a/extensions/standard-processors/processors/DefragmentText.h
+++ b/extensions/standard-processors/processors/DefragmentText.h
@@ -116,7 +116,6 @@ class DefragmentText : public core::ProcessorImpl {
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
void restore(const std::shared_ptr<core::FlowFile>& flowFile) override;
- std::set<core::Connectable*> getOutGoingConnections(const std::string &relationship) override;
protected:
class Buffer {
diff --git a/extensions/standard-processors/processors/ExtractText.h b/extensions/standard-processors/processors/ExtractText.h
index 71ee5fbfb..dd54485a6 100644
--- a/extensions/standard-processors/processors/ExtractText.h
+++ b/extensions/standard-processors/processors/ExtractText.h
@@ -38,10 +38,7 @@ namespace org::apache::nifi::minifi::processors {
class ExtractText : public core::ProcessorImpl {
public:
- explicit ExtractText(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ExtractText>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
// Default maximum bytes to read into an attribute
static constexpr std::string_view DEFAULT_SIZE_LIMIT_STR = "2097152"; // 2 * 1024 * 1024
diff --git a/extensions/standard-processors/processors/GenerateFlowFile.cpp b/extensions/standard-processors/processors/GenerateFlowFile.cpp
index ccc7b8636..24e27170e 100644
--- a/extensions/standard-processors/processors/GenerateFlowFile.cpp
+++ b/extensions/standard-processors/processors/GenerateFlowFile.cpp
@@ -26,12 +26,13 @@
#include <string>
#include <vector>
-#include "core/ProcessContext.h"
+#include "minifi-cpp/core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/Resource.h"
#include "utils/OptionalUtils.h"
#include "utils/gsl.h"
#include "utils/ProcessorConfigUtils.h"
+#include "minifi-cpp/core/ProcessorDescriptor.h"
namespace org::apache::nifi::minifi::processors {
const char *GenerateFlowFile::DATA_FORMAT_TEXT = "Text";
@@ -83,7 +84,7 @@ void GenerateFlowFile::onSchedule(core::ProcessContext& context, core::ProcessSe
| utils::valueOrElse([]() {return false;});
bool is_unique = utils::parseOptionalBoolProperty(context, UniqueFlowFiles).value_or(true);
- const auto custom_text_without_evaluation = getProperty(CustomText.name);
+ const auto custom_text_without_evaluation = context.getProperty(CustomText.name);
const bool has_custom_text = custom_text_without_evaluation.has_value() && !custom_text_without_evaluation->empty();
file_size_ = utils::parseDataSizeProperty(context, FileSize);
diff --git a/extensions/standard-processors/processors/GenerateFlowFile.h b/extensions/standard-processors/processors/GenerateFlowFile.h
index 2e173750c..a3fb0a698 100644
--- a/extensions/standard-processors/processors/GenerateFlowFile.h
+++ b/extensions/standard-processors/processors/GenerateFlowFile.h
@@ -39,10 +39,7 @@ namespace org::apache::nifi::minifi::processors {
class GenerateFlowFile : public core::ProcessorImpl {
public:
- explicit GenerateFlowFile(const std::string_view name, const utils::Identifier& uuid = {}) // NOLINT
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<GenerateFlowFile>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~GenerateFlowFile() override = default;
EXTENSIONAPI static constexpr const char* Description = "This processor creates FlowFiles with random data or custom content. "
diff --git a/extensions/standard-processors/processors/GetFile.h b/extensions/standard-processors/processors/GetFile.h
index 5aa32033a..1e127b8e1 100644
--- a/extensions/standard-processors/processors/GetFile.h
+++ b/extensions/standard-processors/processors/GetFile.h
@@ -54,7 +54,7 @@ struct GetFileRequest {
class GetFileMetrics : public core::ProcessorMetricsImpl {
public:
- explicit GetFileMetrics(const core::Processor& source_processor)
+ explicit GetFileMetrics(const core::ProcessorApi& source_processor)
: core::ProcessorMetricsImpl(source_processor) {
}
diff --git a/extensions/standard-processors/processors/RouteOnAttribute.cpp b/extensions/standard-processors/processors/RouteOnAttribute.cpp
index 2a3d095e2..573634be9 100644
--- a/extensions/standard-processors/processors/RouteOnAttribute.cpp
+++ b/extensions/standard-processors/processors/RouteOnAttribute.cpp
@@ -33,8 +33,8 @@ void RouteOnAttribute::initialize() {
setSupportedRelationships(Relationships);
}
-void RouteOnAttribute::onSchedule(core::ProcessContext &, core::ProcessSessionFactory&) {
- route_properties_ = getDynamicProperties();
+void RouteOnAttribute::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) {
+ route_properties_ = context.getDynamicProperties();
const auto static_relationships = RouteOnAttribute::Relationships;
std::vector<core::RelationshipDefinition> relationships(static_relationships.begin(), static_relationships.end());
@@ -78,7 +78,7 @@ void RouteOnAttribute::onTrigger(core::ProcessContext& context, core::ProcessSes
} catch (const std::exception &e) {
logger_->log_error("Caught exception while updating attributes: type: {}, what: {}", typeid(e).name(), e.what());
session.transfer(flow_file, Failure);
- yield();
+ context.yield();
}
}
diff --git a/extensions/standard-processors/processors/RouteOnAttribute.h b/extensions/standard-processors/processors/RouteOnAttribute.h
index 30fe8ee6c..f1775bab7 100644
--- a/extensions/standard-processors/processors/RouteOnAttribute.h
+++ b/extensions/standard-processors/processors/RouteOnAttribute.h
@@ -35,10 +35,7 @@ namespace org::apache::nifi::minifi::processors {
class RouteOnAttribute : public core::ProcessorImpl {
public:
- explicit RouteOnAttribute(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<RouteOnAttribute>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Routes FlowFiles based on their Attributes using the Attribute Expression Language.\n\n"
"Any number of user-defined dynamic properties can be added, which all support the Attribute Expression Language. Relationships matching the name of the properties will be added.\n"
diff --git a/extensions/standard-processors/processors/SegmentContent.h b/extensions/standard-processors/processors/SegmentContent.h
index 6e4962ad6..364bfbbd8 100644
--- a/extensions/standard-processors/processors/SegmentContent.h
+++ b/extensions/standard-processors/processors/SegmentContent.h
@@ -34,9 +34,7 @@ namespace org::apache::nifi::minifi::processors {
class SegmentContent final : public core::ProcessorImpl {
public:
- explicit SegmentContent(const std::string_view name, const utils::Identifier& uuid = {}) : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<SegmentContent>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr auto Description = "Segments a FlowFile into multiple smaller segments on byte boundaries.";
diff --git a/extensions/standard-processors/processors/SplitText.h b/extensions/standard-processors/processors/SplitText.h
index b461aad9b..93f9f82a2 100644
--- a/extensions/standard-processors/processors/SplitText.h
+++ b/extensions/standard-processors/processors/SplitText.h
@@ -87,10 +87,7 @@ class LineReader {
class SplitText : public core::ProcessorImpl {
public:
- explicit SplitText(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<SplitText>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Splits a text file into multiple smaller text files on line boundaries limited by maximum number of lines or total size of fragment. "
"Each output split file will contain no more than the configured number of lines or bytes. If both Line Split Count and Maximum Fragment Size are specified, the split occurs at whichever "
diff --git a/extensions/standard-processors/processors/UpdateAttribute.h b/extensions/standard-processors/processors/UpdateAttribute.h
index 2cd1ba2d0..e877b0669 100644
--- a/extensions/standard-processors/processors/UpdateAttribute.h
+++ b/extensions/standard-processors/processors/UpdateAttribute.h
@@ -35,10 +35,7 @@ namespace org::apache::nifi::minifi::processors {
class UpdateAttribute : public core::ProcessorImpl {
public:
- explicit UpdateAttribute(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<UpdateAttribute>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "This processor updates the attributes of a FlowFile using properties that are added by the user. "
"This allows you to set default attribute changes that affect every FlowFile going through the processor, equivalent to the \"basic\" usage in Apache NiFi.";
diff --git a/libminifi/include/ForwardingNode.h b/libminifi/include/ForwardingNode.h
index d20468b27..ffa8e7713 100644
--- a/libminifi/include/ForwardingNode.h
+++ b/libminifi/include/ForwardingNode.h
@@ -22,15 +22,14 @@
#include <utility>
#include "core/logging/LoggerFactory.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
+#include "minifi-cpp/core/ProcessorDescriptor.h"
namespace org::apache::nifi::minifi {
class ForwardingNode : public core::ProcessorImpl {
public:
- ForwardingNode(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger) : ProcessorImpl(name, uuid), logger_(std::move(logger)) {
- strategy_ = core::SchedulingStrategy::EVENT_DRIVEN;
- }
+ ForwardingNode(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger) : ProcessorImpl(name, uuid), logger_(std::move(logger)) {}
ForwardingNode(std::string_view name, std::shared_ptr<core::logging::Logger> logger) : ProcessorImpl(name), logger_(std::move(logger)) {}
MINIFIAPI static constexpr auto Properties = std::array<core::PropertyReference, 0>{};
diff --git a/libminifi/include/RemoteProcessorGroupPort.h b/libminifi/include/RemoteProcessorGroupPort.h
index 52a4e2bcb..0f19721ec 100644
--- a/libminifi/include/RemoteProcessorGroupPort.h
+++ b/libminifi/include/RemoteProcessorGroupPort.h
@@ -29,7 +29,7 @@
#include "http/BaseHTTPClient.h"
#include "concurrentqueue.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -212,7 +212,7 @@ class RemoteProcessorGroupPort : public core::ProcessorImpl {
}
std::unique_ptr<sitetosite::SiteToSiteClient> getNextProtocol(bool create);
- void returnProtocol(std::unique_ptr<sitetosite::SiteToSiteClient> protocol);
+ void returnProtocol(core::ProcessContext& context, std::unique_ptr<sitetosite::SiteToSiteClient> protocol);
moodycamel::ConcurrentQueue<std::unique_ptr<sitetosite::SiteToSiteClient>> available_protocols_;
diff --git a/minifi-api/include/minifi-cpp/core/Processor.h b/libminifi/include/core/Processor.h
similarity index 80%
copy from minifi-api/include/minifi-cpp/core/Processor.h
copy to libminifi/include/core/Processor.h
index ded0f7570..8d4904827 100644
--- a/minifi-api/include/minifi-cpp/core/Processor.h
+++ b/libminifi/include/core/Processor.h
@@ -23,17 +23,16 @@
#include <unordered_set>
#include <unordered_map>
-#include "ConfigurableComponent.h"
-#include "Connectable.h"
-#include "Property.h"
-#include "DynamicProperty.h"
-#include "Core.h"
+#include "minifi-cpp/core/ConfigurableComponent.h"
+#include "minifi-cpp/core/Connectable.h"
+#include "minifi-cpp/core/Property.h"
+#include "minifi-cpp/core/DynamicProperty.h"
+#include "minifi-cpp/core/Core.h"
#include "minifi-cpp/core/Annotation.h"
-#include "Scheduling.h"
+#include "minifi-cpp/core/Scheduling.h"
#include "minifi-cpp/core/state/nodes/MetricsBase.h"
-#include "ProcessorMetrics.h"
+#include "minifi-cpp/core/ProcessorMetrics.h"
#include "utils/gsl.h"
-#include "core/logging/Logger.h"
namespace org::apache::nifi::minifi {
@@ -64,7 +63,6 @@ class Processor : public virtual Connectable, public virtual ConfigurableCompone
virtual void setPenalizationPeriod(std::chrono::milliseconds period) = 0;
virtual bool isSingleThreaded() const = 0;
virtual std::string getProcessorType() const = 0;
- virtual void setTriggerWhenEmpty(bool value) = 0;
virtual bool getTriggerWhenEmpty() const = 0;
virtual uint8_t getActiveTasks() const = 0;
virtual void incrementActiveTasks() = 0;
@@ -76,7 +74,6 @@ class Processor : public virtual Connectable, public virtual ConfigurableCompone
virtual void clearYield() = 0;
virtual std::chrono::steady_clock::time_point getYieldExpirationTime() const = 0;
virtual std::chrono::steady_clock::duration getYieldTime() const = 0;
- virtual bool flowFilesOutGoingFull() const = 0;
virtual bool addConnection(Connectable* connection) = 0;
virtual void triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) = 0;
virtual void trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) = 0;
@@ -87,15 +84,6 @@ class Processor : public virtual Connectable, public virtual ConfigurableCompone
virtual void validateAnnotations() const = 0;
virtual annotation::Input getInputRequirement() const = 0;
virtual gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const = 0;
- virtual std::string getProcessGroupUUIDStr() const = 0;
- virtual void setProcessGroupUUIDStr(const std::string &uuid) = 0;
- virtual std::string getProcessGroupName() const = 0;
- virtual void setProcessGroupName(const std::string &name) = 0;
- virtual std::string getProcessGroupPath() const = 0;
- virtual void setProcessGroupPath(const std::string &path) = 0;
- virtual logging::LOG_LEVEL getLogBulletinLevel() const = 0;
- virtual void setLogBulletinLevel(logging::LOG_LEVEL level) = 0;
- virtual void setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback) = 0;
virtual void updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force = false) = 0;
virtual const std::unordered_map<Connection*, std::unordered_set<Processor*>>& reachable_processors() const = 0;
diff --git a/utils/include/core/Processor.h b/libminifi/include/core/ProcessorProxy.h
similarity index 67%
copy from utils/include/core/Processor.h
copy to libminifi/include/core/ProcessorProxy.h
index 11c779b00..5ea8931ce 100644
--- a/utils/include/core/Processor.h
+++ b/libminifi/include/core/ProcessorProxy.h
@@ -42,22 +42,9 @@
#include "utils/gsl.h"
#include "utils/Id.h"
#include "minifi-cpp/core/OutputAttributeDefinition.h"
+#include "Processor.h"
#include "minifi-cpp/core/Processor.h"
-#define ADD_GET_PROCESSOR_NAME \
- std::string getProcessorType() const override { \
- auto class_name = org::apache::nifi::minifi::core::className<decltype(*this)>(); \
- auto splitted = org::apache::nifi::minifi::utils::string::split(class_name, "::"); \
- return splitted[splitted.size() - 1]; \
- }
-
-#define ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS \
- bool supportsDynamicProperties() const override { return SupportsDynamicProperties; } \
- bool supportsDynamicRelationships() const override { return SupportsDynamicRelationships; } \
- minifi::core::annotation::Input getInputRequirement() const override { return InputRequirement; } \
- bool isSingleThreaded() const override { return IsSingleThreaded; } \
- ADD_GET_PROCESSOR_NAME
-
namespace org::apache::nifi::minifi {
class Connection;
@@ -72,17 +59,17 @@ constexpr std::chrono::microseconds MINIMUM_SCHEDULING_PERIOD{30};
#define BUILDING_DLL 1
-class ProcessorImpl : public virtual Processor, public ConnectableImpl, public ConfigurableComponentImpl {
+class ProcessorProxy : public virtual Processor, public ConnectableImpl, public ConfigurableComponentImpl {
public:
- ProcessorImpl(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<ProcessorMetrics> metrics = nullptr);
- explicit ProcessorImpl(std::string_view name, std::shared_ptr<ProcessorMetrics> metrics = nullptr);
+ ProcessorProxy(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<ProcessorApi> impl);
+ explicit ProcessorProxy(std::string_view name, std::unique_ptr<ProcessorApi> impl);
- ProcessorImpl(const ProcessorImpl& parent) = delete;
- ProcessorImpl& operator=(const ProcessorImpl& parent) = delete;
+ ProcessorProxy(const ProcessorProxy& parent) = delete;
+ ProcessorProxy& operator=(const ProcessorProxy& parent) = delete;
bool isRunning() const override;
- ~ProcessorImpl() override;
+ ~ProcessorProxy() override;
void setScheduledState(ScheduledState state) override;
@@ -136,16 +123,16 @@ class ProcessorImpl : public virtual Processor, public ConnectableImpl, public C
void setMaxConcurrentTasks(uint8_t tasks) override;
- bool isSingleThreaded() const override = 0;
-
- std::string getProcessorType() const override = 0;
+ bool isSingleThreaded() const override {
+ return impl_->isSingleThreaded();
+ }
- void setTriggerWhenEmpty(bool value) override {
- _triggerWhenEmpty = value;
+ std::string getProcessorType() const override {
+ return impl_->getProcessorType();
}
bool getTriggerWhenEmpty() const override {
- return (_triggerWhenEmpty);
+ return impl_->getTriggerWhenEmpty();
}
uint8_t getActiveTasks() const override {
@@ -165,30 +152,6 @@ class ProcessorImpl : public virtual Processor, public ConnectableImpl, public C
active_tasks_ = 0;
}
- std::string getProcessGroupUUIDStr() const override {
- return process_group_uuid_;
- }
-
- void setProcessGroupUUIDStr(const std::string &uuid) override {
- process_group_uuid_ = uuid;
- }
-
- std::string getProcessGroupName() const override {
- return process_group_name_;
- }
-
- void setProcessGroupName(const std::string &name) override {
- process_group_name_ = name;
- }
-
- std::string getProcessGroupPath() const override {
- return process_group_path_;
- }
-
- void setProcessGroupPath(const std::string &path) override {
- process_group_path_ = path;
- }
-
void yield() override;
void yield(std::chrono::steady_clock::duration delta_time) override;
@@ -199,8 +162,6 @@ class ProcessorImpl : public virtual Processor, public ConnectableImpl, public C
std::chrono::steady_clock::time_point getYieldExpirationTime() const override { return yield_expiration_; }
std::chrono::steady_clock::duration getYieldTime() const override;
- // Whether flow file queue full in any of the outgoing connection
- bool flowFilesOutGoingFull() const override;
bool addConnection(Connectable* connection) override;
@@ -208,19 +169,22 @@ class ProcessorImpl : public virtual Processor, public ConnectableImpl, public C
return !isRunning();
}
- void initialize() override {
- }
+ void initialize() override;
void triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) override;
void trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) override;
- void onTrigger(ProcessContext&, ProcessSession&) override {}
+ void onTrigger(ProcessContext& context, ProcessSession& session) override {
+ impl_->onTrigger(context, session);
+ }
- void onSchedule(ProcessContext&, ProcessSessionFactory&) override {}
+ void onSchedule(ProcessContext& context, ProcessSessionFactory& session_factory) override {
+ impl_->onSchedule(context, session_factory);
+ }
// Hook executed when onSchedule fails (throws). Configuration should be reset in this
void onUnSchedule() override {
- notifyStop();
+ impl_->onUnSchedule();
}
// Check all incoming connections for work
@@ -232,34 +196,31 @@ class ProcessorImpl : public virtual Processor, public ConnectableImpl, public C
void validateAnnotations() const override;
- annotation::Input getInputRequirement() const override = 0;
-
- state::response::SharedResponseNode getResponseNode() override {
- return metrics_;
+ annotation::Input getInputRequirement() const override {
+ return impl_->getInputRequirement();
}
- gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const override {
- return metrics_;
+ [[nodiscard]] bool supportsDynamicProperties() const override {
+ return impl_->supportsDynamicProperties();
}
- logging::LOG_LEVEL getLogBulletinLevel() const override {
- return log_bulletin_level_;
+ [[nodiscard]] bool supportsDynamicRelationships() const override {
+ return impl_->supportsDynamicRelationships();
}
- void setLogBulletinLevel(logging::LOG_LEVEL level) override {
- log_bulletin_level_ = level;
+ state::response::SharedResponseNode getResponseNode() override {
+ return getMetrics();
}
- void setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback) override;
+ gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const override {
+ return impl_->getMetrics();
+ }
static constexpr auto DynamicProperties = std::array<DynamicProperty, 0>{};
static constexpr auto OutputAttributes = std::array<OutputAttributeReference, 0>{};
protected:
- virtual void notifyStop() {
- }
-
std::atomic<ScheduledState> state_;
std::atomic<std::chrono::steady_clock::duration> scheduling_period_;
@@ -267,13 +228,10 @@ class ProcessorImpl : public virtual Processor, public ConnectableImpl, public C
std::atomic<std::chrono::steady_clock::duration> yield_period_;
std::atomic<uint8_t> active_tasks_;
- std::atomic<bool> _triggerWhenEmpty;
std::string cron_period_;
- gsl::not_null<std::shared_ptr<ProcessorMetrics>> metrics_;
std::shared_ptr<logging::Logger> logger_;
- logging::LOG_LEVEL log_bulletin_level_ = logging::LOG_LEVEL::warn;
private:
mutable std::mutex mutex_;
@@ -296,9 +254,7 @@ class ProcessorImpl : public virtual Processor, public ConnectableImpl, public C
// an outgoing connection allows us to reach these nodes
std::unordered_map<Connection*, std::unordered_set<Processor*>> reachable_processors_;
- std::string process_group_uuid_;
- std::string process_group_name_;
- std::string process_group_path_;
+ std::unique_ptr<ProcessorApi> impl_;
};
} // namespace core
diff --git a/libminifi/include/core/state/nodes/FlowInformation.h b/libminifi/include/core/state/nodes/FlowInformation.h
index e0f69b2ca..cacd4dfcb 100644
--- a/libminifi/include/core/state/nodes/FlowInformation.h
+++ b/libminifi/include/core/state/nodes/FlowInformation.h
@@ -52,7 +52,7 @@ class FlowVersion : public DeviceInformation {
return "FlowVersion";
}
- std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const override {
+ std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const {
std::lock_guard<std::mutex> lock(guard);
return identifier;
}
diff --git a/libminifi/include/core/state/nodes/MetricsBase.h b/libminifi/include/core/state/nodes/MetricsBase.h
index 555b06d70..0cc9ebd97 100644
--- a/libminifi/include/core/state/nodes/MetricsBase.h
+++ b/libminifi/include/core/state/nodes/MetricsBase.h
@@ -54,7 +54,7 @@ class ObjectNode : public ResponseNodeImpl {
}
std::string getName() const override {
- return ConnectableImpl::getName();
+ return CoreComponentImpl::getName();
}
std::vector<SerializedResponseNode> serialize() override;
diff --git a/libminifi/src/RemoteProcessorGroupPort.cpp b/libminifi/src/RemoteProcessorGroupPort.cpp
index 144a8f5ab..3f1b99179 100644
--- a/libminifi/src/RemoteProcessorGroupPort.cpp
+++ b/libminifi/src/RemoteProcessorGroupPort.cpp
@@ -88,10 +88,10 @@ std::unique_ptr<sitetosite::SiteToSiteClient> RemoteProcessorGroupPort::getNextP
return nextProtocol;
}
-void RemoteProcessorGroupPort::returnProtocol(std::unique_ptr<sitetosite::SiteToSiteClient> return_protocol) {
+void RemoteProcessorGroupPort::returnProtocol(core::ProcessContext& context, std::unique_ptr<sitetosite::SiteToSiteClient> return_protocol) {
auto count = peers_.size();
- if (max_concurrent_tasks_ > count)
- count = max_concurrent_tasks_;
+ if (context.getProcessor().getMaxConcurrentTasks() > count)
+ count = context.getProcessor().getMaxConcurrentTasks();
if (available_protocols_.size_approx() >= count) {
logger_->log_debug("not enqueueing protocol {}", getUUIDStr());
// let the memory be freed
@@ -140,8 +140,8 @@ void RemoteProcessorGroupPort::onSchedule(core::ProcessContext& context, core::P
// populate the site2site protocol for load balancing between them
if (!peers_.empty()) {
auto count = peers_.size();
- if (max_concurrent_tasks_ > count)
- count = max_concurrent_tasks_;
+ if (context.getProcessor().getMaxConcurrentTasks() > count)
+ count = context.getProcessor().getMaxConcurrentTasks();
for (uint32_t i = 0; i < count; i++) {
std::unique_ptr<sitetosite::SiteToSiteClient> nextProtocol = nullptr;
sitetosite::SiteToSiteClientConfiguration config(peers_[this->peer_index_].getPeer(), this->getInterface(), client_type_);
@@ -155,7 +155,7 @@ void RemoteProcessorGroupPort::onSchedule(core::ProcessContext& context, core::P
config.setIdleTimeout(idle_timeout_);
nextProtocol = sitetosite::createClient(config);
logger_->log_trace("Created client, moving into available protocols");
- returnProtocol(std::move(nextProtocol));
+ returnProtocol(context, std::move(nextProtocol));
}
} else {
// we don't have any peers
@@ -203,7 +203,7 @@ void RemoteProcessorGroupPort::onTrigger(core::ProcessContext& context, core::Pr
context.yield();
}
- returnProtocol(std::move(protocol_));
+ returnProtocol(context, std::move(protocol_));
return;
} catch (const minifi::Exception &) {
context.yield();
diff --git a/libminifi/src/core/ClassLoader.cpp b/libminifi/src/core/ClassLoader.cpp
index 985d4cb72..9d56c36e5 100644
--- a/libminifi/src/core/ClassLoader.cpp
+++ b/libminifi/src/core/ClassLoader.cpp
@@ -23,6 +23,7 @@
#include "core/logging/LoggerFactory.h"
#include "range/v3/action/sort.hpp"
#include "range/v3/action/unique.hpp"
+#include "core/ProcessorProxy.h"
namespace org {
namespace apache {
@@ -38,6 +39,8 @@ class ClassLoaderImpl : public ClassLoader {
void registerClass(const std::string &clazz, std::unique_ptr<ObjectFactory> factory) override;
+ void registerClass(const std::string &clazz, std::unique_ptr<ProcessorFactory> factory) override;
+
void unregisterClass(const std::string& clazz) override;
std::optional<std::string> getGroupForClass(const std::string &class_name) const override;
@@ -96,6 +99,47 @@ void ClassLoaderImpl::registerClass(const std::string &clazz, std::unique_ptr<Ob
loaded_factories_.insert(std::make_pair(clazz, std::move(factory)));
}
+namespace {
+class ProcessorFactoryWrapper : public ObjectFactoryImpl {
+ public:
+ explicit ProcessorFactoryWrapper(std::unique_ptr<ProcessorFactory> factory)
+ : ObjectFactoryImpl(factory->getGroupName()),
+ factory_(std::move(factory)) {}
+
+ std::unique_ptr<CoreComponent> create(const std::string &name) override {
+ return std::unique_ptr<CoreComponent>{createRaw(name)};
+ }
+
+ std::unique_ptr<CoreComponent> create(const std::string &name, const utils::Identifier &uuid) override {
+ return std::unique_ptr<CoreComponent>{createRaw(name, uuid)};
+ }
+
+ CoreComponent* createRaw(const std::string &name) override {
+ return createRaw(name, utils::IdGenerator::getIdGenerator()->generate());
+ }
+
+ CoreComponent* createRaw(const std::string &name, const utils::Identifier &uuid) override {
+ auto logger = logging::LoggerFactoryBase::getAliasedLogger(getClassName(), uuid);
+ return new ProcessorProxy(name, uuid, factory_->create({.uuid = uuid, .name = name, .logger = logger}));
+ }
+
+ std::string getGroupName() const override {
+ return factory_->getGroupName();
+ }
+
+ std::string getClassName() override {
+ return factory_->getClassName();
+ }
+
+ private:
+ std::unique_ptr<ProcessorFactory> factory_;
+};
+}
+
+void ClassLoaderImpl::registerClass(const std::string &clazz, std::unique_ptr<ProcessorFactory> factory) {
+ registerClass(clazz, std::make_unique<ProcessorFactoryWrapper>(std::move(factory)));
+}
+
void ClassLoaderImpl::unregisterClass(const std::string& clazz) {
std::lock_guard<std::mutex> lock(internal_mutex_);
if (loaded_factories_.erase(clazz) == 0) {
diff --git a/libminifi/src/core/FlowConfiguration.cpp b/libminifi/src/core/FlowConfiguration.cpp
index 761aafdd8..17285480c 100644
--- a/libminifi/src/core/FlowConfiguration.cpp
+++ b/libminifi/src/core/FlowConfiguration.cpp
@@ -26,6 +26,7 @@
#include "processors/ProcessorUtils.h"
#include "utils/StringUtils.h"
#include "utils/file/FileUtils.h"
+#include "core/ProcessorProxy.h"
namespace org::apache::nifi::minifi::core {
diff --git a/libminifi/src/core/ProcessContext.cpp b/libminifi/src/core/ProcessContext.cpp
new file mode 100644
index 000000000..63bb8bc8b
--- /dev/null
+++ b/libminifi/src/core/ProcessContext.cpp
@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "core/ProcessContext.h"
+#include "core/Processor.h"
+
+namespace org::apache::nifi::minifi::core {
+
+namespace {
+
+class StandardProcessorInfo : public ProcessorInfo {
+ public:
+ explicit StandardProcessorInfo(Processor& proc): proc_(proc) {}
+
+ std::string getName() const override {return proc_.getName();}
+ utils::Identifier getUUID() const override {return proc_.getUUID();}
+ std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const override {return proc_.getFlowIdentifier();}
+ std::map<std::string, core::Property> getSupportedProperties() const override {return proc_.getSupportedProperties();}
+ nonstd::expected<PropertyReference, std::error_code> getPropertyReference(std::string_view name) const override {return proc_.getPropertyReference(name);}
+
+ private:
+ Processor& proc_;
+};
+
+} // namespace
+
+ProcessContextImpl::ProcessContextImpl(
+ Processor& processor, controller::ControllerServiceProvider* controller_service_provider, const std::shared_ptr<core::Repository>& repo,
+ const std::shared_ptr<core::Repository>& flow_repo, const std::shared_ptr<core::ContentRepository>& content_repo)
+ : VariableRegistryImpl(Configure::create()),
+ controller_service_provider_(controller_service_provider),
+ flow_repo_(flow_repo),
+ content_repo_(content_repo),
+ processor_(processor),
+ logger_(logging::LoggerFactory<ProcessContext>::getLogger()),
+ configure_(minifi::Configure::create()),
+ initialized_(false) {
+ repo_ = repo;
+ state_storage_ = getStateStorage(logger_, controller_service_provider_, nullptr);
+}
+
+ProcessContextImpl::ProcessContextImpl(
+ Processor& processor, controller::ControllerServiceProvider* controller_service_provider, const std::shared_ptr<core::Repository>& repo,
+ const std::shared_ptr<core::Repository>& flow_repo, const std::shared_ptr<minifi::Configure>& configuration,
+ const std::shared_ptr<core::ContentRepository>& content_repo)
+ : VariableRegistryImpl(configuration),
+ controller_service_provider_(controller_service_provider),
+ flow_repo_(flow_repo),
+ content_repo_(content_repo),
+ processor_(processor),
+ logger_(logging::LoggerFactory<ProcessContext>::getLogger()),
+ configure_(configuration),
+ initialized_(false) {
+ repo_ = repo;
+ state_storage_ = getStateStorage(logger_, controller_service_provider_, configuration);
+ if (!configure_) { configure_ = minifi::Configure::create(); }
+ info_ = std::make_unique<StandardProcessorInfo>(processor);
+}
+
+std::vector<std::string> ProcessContextImpl::getDynamicPropertyKeys() const { return processor_.getDynamicPropertyKeys(); }
+
+std::map<std::string, std::string> ProcessContextImpl::getDynamicProperties(const FlowFile*) const { return processor_.getDynamicProperties(); }
+
+bool ProcessContextImpl::isAutoTerminated(Relationship relationship) const { return processor_.isAutoTerminated(relationship); }
+
+uint8_t ProcessContextImpl::getMaxConcurrentTasks() const { return processor_.getMaxConcurrentTasks(); }
+
+void ProcessContextImpl::yield() { processor_.yield(); }
+
+nonstd::expected<std::string, std::error_code> ProcessContextImpl::getProperty(const std::string_view name, const FlowFile* const) const {
+ return getProcessor().getProperty(name);
+}
+
+nonstd::expected<void, std::error_code> ProcessContextImpl::setProperty(const std::string_view name, std::string value) {
+ return getProcessor().setProperty(name, std::move(value));
+}
+
+nonstd::expected<void, std::error_code> ProcessContextImpl::clearProperty(const std::string_view name) {
+ return getProcessor().clearProperty(name);
+}
+
+nonstd::expected<std::string, std::error_code> ProcessContextImpl::getDynamicProperty(const std::string_view name, const FlowFile* const) const {
+ return getProcessor().getDynamicProperty(name);
+}
+
+nonstd::expected<void, std::error_code> ProcessContextImpl::setDynamicProperty(std::string name, std::string value) {
+ return getProcessor().setDynamicProperty(std::move(name), std::move(value));
+}
+
+StateManager* ProcessContextImpl::getStateManager() {
+ if (state_storage_ == nullptr) { return nullptr; }
+ if (!state_manager_) { state_manager_ = state_storage_->getStateManager(processor_); }
+ return state_manager_.get();
+}
+
+} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/src/core/ProcessSession.cpp b/libminifi/src/core/ProcessSession.cpp
index e4b8838bf..37706642c 100644
--- a/libminifi/src/core/ProcessSession.cpp
+++ b/libminifi/src/core/ProcessSession.cpp
@@ -34,6 +34,7 @@
#include "io/StreamSlice.h"
#include "io/StreamPipe.h"
#include "utils/gsl.h"
+#include "core/Processor.h"
/* This implementation is only for native Windows systems. */
#if (defined _WIN32 || defined __WIN32__) && !defined __CYGWIN__
diff --git a/utils/src/core/Processor.cpp b/libminifi/src/core/Processor.cpp
similarity index 81%
copy from utils/src/core/Processor.cpp
copy to libminifi/src/core/Processor.cpp
index 95feac9f8..538c265fb 100644
--- a/utils/src/core/Processor.cpp
+++ b/libminifi/src/core/Processor.cpp
@@ -32,26 +32,26 @@
#include "core/logging/LoggerFactory.h"
#include "minifi-cpp/core/ProcessorConfig.h"
#include "minifi-cpp/core/ProcessContext.h"
+#include "minifi-cpp/core/ProcessorDescriptor.h"
#include "minifi-cpp/core/ProcessSessionFactory.h"
#include "utils/gsl.h"
#include "range/v3/algorithm/any_of.hpp"
#include "fmt/format.h"
#include "Exception.h"
+#include "core/ProcessorProxy.h"
#include "core/ProcessorMetrics.h"
using namespace std::literals::chrono_literals;
namespace org::apache::nifi::minifi::core {
-ProcessorImpl::ProcessorImpl(std::string_view name, std::shared_ptr<ProcessorMetrics> metrics)
+ProcessorProxy::ProcessorProxy(std::string_view name, std::unique_ptr<ProcessorApi> impl)
: ConnectableImpl(name),
state_(DISABLED),
scheduling_period_(MINIMUM_SCHEDULING_PERIOD),
run_duration_(DEFAULT_RUN_DURATION),
yield_period_(DEFAULT_YIELD_PERIOD_SECONDS),
active_tasks_(0),
- _triggerWhenEmpty(false),
- metrics_(metrics ? std::move(metrics) : std::make_shared<ProcessorMetricsImpl>(*this)),
logger_(logging::LoggerFactory<Processor>::getLogger(uuid_)) {
has_work_.store(false);
// Setup the default values
@@ -62,15 +62,13 @@ ProcessorImpl::ProcessorImpl(std::string_view name, std::shared_ptr<ProcessorMet
logger_->log_debug("Processor {} created UUID {}", name_, getUUIDStr());
}
-ProcessorImpl::ProcessorImpl(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<ProcessorMetrics> metrics)
+ProcessorProxy::ProcessorProxy(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<ProcessorApi> impl)
: ConnectableImpl(name, uuid),
state_(DISABLED),
scheduling_period_(MINIMUM_SCHEDULING_PERIOD),
run_duration_(DEFAULT_RUN_DURATION),
yield_period_(DEFAULT_YIELD_PERIOD_SECONDS),
active_tasks_(0),
- _triggerWhenEmpty(false),
- metrics_(metrics ? std::move(metrics) : std::make_shared<ProcessorMetricsImpl>(*this)),
logger_(logging::LoggerFactory<Processor>::getLogger(uuid_)) {
has_work_.store(false);
// Setup the default values
@@ -81,22 +79,22 @@ ProcessorImpl::ProcessorImpl(std::string_view name, const utils::Identifier& uui
logger_->log_debug("Processor {} created with uuid {}", name_, getUUIDStr());
}
-ProcessorImpl::~ProcessorImpl() {
+ProcessorProxy::~ProcessorProxy() {
logger_->log_debug("Destroying processor {} with uuid {}", name_, getUUIDStr());
}
-bool ProcessorImpl::isRunning() const {
+bool ProcessorProxy::isRunning() const {
return (state_ == RUNNING && active_tasks_ > 0);
}
-void ProcessorImpl::setScheduledState(ScheduledState state) {
+void ProcessorProxy::setScheduledState(ScheduledState state) {
state_ = state;
if (state == STOPPED) {
- notifyStop();
+ impl_->notifyStop();
}
}
-bool ProcessorImpl::addConnection(Connectable* conn) {
+bool ProcessorProxy::addConnection(Connectable* conn) {
enum class SetAs{
NONE,
OUTPUT,
@@ -166,24 +164,9 @@ bool ProcessorImpl::addConnection(Connectable* conn) {
return result != SetAs::NONE;
}
-bool ProcessorImpl::flowFilesOutGoingFull() const {
- std::lock_guard<std::mutex> lock(mutex_);
-
- for (const auto& [_name, existed_connection] : outgoing_connections_) {
- if (ranges::any_of(existed_connection, [](const Connectable* conn) {
- const auto connection = dynamic_cast<const Connection*>(conn);
- return connection && connection->backpressureThresholdReached();
- })) {
- return true;
- }
- }
-
- return false;
-}
-
-void ProcessorImpl::triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) {
+void ProcessorProxy::triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) {
const auto process_session = session_factory->createSession();
- process_session->setMetrics(metrics_);
+ process_session->setMetrics(getMetrics());
try {
trigger(context, process_session);
process_session->commit();
@@ -199,14 +182,14 @@ void ProcessorImpl::triggerAndCommit(const std::shared_ptr<ProcessContext>& cont
}
}
-void ProcessorImpl::trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) {
- ++metrics_->invocations();
+void ProcessorProxy::trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) {
+ ++impl_->getMetrics()->iterations();
const auto start = std::chrono::steady_clock::now();
onTrigger(*context, *process_session);
- metrics_->addLastOnTriggerRuntime(std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - start));
+ impl_->getMetrics()->addLastOnTriggerRuntime(std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - start));
}
-bool ProcessorImpl::isWorkAvailable() {
+bool ProcessorProxy::isWorkAvailable() {
// We have work if any incoming connection has work
std::lock_guard<std::mutex> lock(mutex_);
bool hasWork = false;
@@ -228,11 +211,11 @@ bool ProcessorImpl::isWorkAvailable() {
getCurrentExceptionTypeName());
}
- return hasWork;
+ return hasWork || impl_->isWorkAvailable();
}
// must hold the graphMutex
-void ProcessorImpl::updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force) {
+void ProcessorProxy::updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force) {
bool didChange = force;
for (auto& outIt : outgoing_connections_) {
for (auto& outConn : outIt.second) {
@@ -272,7 +255,7 @@ void ProcessorImpl::updateReachability(const std::lock_guard<std::mutex>& graph_
}
}
-bool ProcessorImpl::partOfCycle(Connection* conn) {
+bool ProcessorProxy::partOfCycle(Connection* conn) {
auto source = dynamic_cast<Processor*>(conn->getSource());
if (!source) {
return false;
@@ -284,7 +267,7 @@ bool ProcessorImpl::partOfCycle(Connection* conn) {
return it->second.contains(source);
}
-bool ProcessorImpl::isThrottledByBackpressure() const {
+bool ProcessorProxy::isThrottledByBackpressure() const {
bool isThrottledByOutgoing = ranges::any_of(outgoing_connections_, [](auto& name_connection_set_pair) {
return ranges::any_of(name_connection_set_pair.second, [](auto& connectable) {
auto connection = dynamic_cast<Connection*>(connectable);
@@ -298,7 +281,7 @@ bool ProcessorImpl::isThrottledByBackpressure() const {
return isThrottledByOutgoing && !isForcedByIncomingCycle;
}
-Connectable* ProcessorImpl::pickIncomingConnection() {
+Connectable* ProcessorProxy::pickIncomingConnection() {
std::lock_guard<std::mutex> rel_guard(relationship_mutex_);
auto beginIt = incoming_connections_Iter;
@@ -318,7 +301,7 @@ Connectable* ProcessorImpl::pickIncomingConnection() {
return getNextIncomingConnectionImpl(rel_guard);
}
-void ProcessorImpl::validateAnnotations() const {
+void ProcessorProxy::validateAnnotations() const {
switch (getInputRequirement()) {
case annotation::Input::INPUT_REQUIRED: {
if (!hasIncomingConnections()) {
@@ -338,7 +321,7 @@ void ProcessorImpl::validateAnnotations() const {
}
}
-void ProcessorImpl::setMaxConcurrentTasks(const uint8_t tasks) {
+void ProcessorProxy::setMaxConcurrentTasks(const uint8_t tasks) {
if (isSingleThreaded() && tasks > 1) {
logger_->log_warn("Processor {} can not be run in parallel, its \"max concurrent tasks\" value is too high. "
"It was set to 1 from {}.", name_, tasks);
@@ -349,28 +332,48 @@ void ProcessorImpl::setMaxConcurrentTasks(const uint8_t tasks) {
max_concurrent_tasks_ = tasks;
}
-void ProcessorImpl::yield() {
+void ProcessorProxy::yield() {
yield_expiration_ = std::chrono::steady_clock::now() + yield_period_.load();
}
-void ProcessorImpl::yield(std::chrono::steady_clock::duration delta_time) {
+void ProcessorProxy::yield(std::chrono::steady_clock::duration delta_time) {
yield_expiration_ = std::chrono::steady_clock::now() + delta_time;
}
-bool ProcessorImpl::isYield() {
+bool ProcessorProxy::isYield() {
return getYieldTime() > 0ms;
}
-void ProcessorImpl::clearYield() {
+void ProcessorProxy::clearYield() {
yield_expiration_ = std::chrono::steady_clock::time_point();
}
-std::chrono::steady_clock::duration ProcessorImpl::getYieldTime() const {
+std::chrono::steady_clock::duration ProcessorProxy::getYieldTime() const {
return std::max(yield_expiration_.load()-std::chrono::steady_clock::now(), std::chrono::steady_clock::duration{0});
}
-void ProcessorImpl::setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback) {
- logger_->setLogCallback(callback);
+namespace {
+
+class ProcessorDescriptorImpl : public ProcessorDescriptor {
+ public:
+ explicit ProcessorDescriptorImpl(ProcessorProxy* impl): impl_(impl) {}
+ void setSupportedRelationships(std::span<const RelationshipDefinition> relationships) override {
+ impl_->setSupportedRelationships(relationships);
+ }
+
+ void setSupportedProperties(std::span<const PropertyReference> properties) {
+ impl_->setSupportedProperties(properties);
+ }
+
+ private:
+ ProcessorProxy* impl_;
+};
+
+} // namespace
+
+void ProcessorProxy::initialize() {
+ ProcessorDescriptorImpl self{this};
+ impl_->initialize(self);
}
} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/test/libtest/unit/DummyProcessor.h b/libminifi/test/libtest/unit/DummyProcessor.h
index 162d69a55..336c9d2e0 100644
--- a/libminifi/test/libtest/unit/DummyProcessor.h
+++ b/libminifi/test/libtest/unit/DummyProcessor.h
@@ -20,7 +20,7 @@
#include <string_view>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "agent/agent_docs.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/libminifi/test/libtest/unit/ReadFromFlowFileTestProcessor.h b/libminifi/test/libtest/unit/ReadFromFlowFileTestProcessor.h
index a1fdd7f28..ac055b5ef 100644
--- a/libminifi/test/libtest/unit/ReadFromFlowFileTestProcessor.h
+++ b/libminifi/test/libtest/unit/ReadFromFlowFileTestProcessor.h
@@ -22,7 +22,7 @@
#include <utility>
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/RelationshipDefinition.h"
diff --git a/libminifi/test/libtest/unit/StatefulProcessor.h b/libminifi/test/libtest/unit/StatefulProcessor.h
index 5659b1692..16aa18c12 100644
--- a/libminifi/test/libtest/unit/StatefulProcessor.h
+++ b/libminifi/test/libtest/unit/StatefulProcessor.h
@@ -20,7 +20,7 @@
#include <memory>
#include <utility>
#include <vector>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/StateManager.h"
namespace org::apache::nifi::minifi::processors {
diff --git a/libminifi/test/libtest/unit/TestBase.cpp b/libminifi/test/libtest/unit/TestBase.cpp
index bec3fc9c0..115fea1af 100644
--- a/libminifi/test/libtest/unit/TestBase.cpp
+++ b/libminifi/test/libtest/unit/TestBase.cpp
@@ -476,7 +476,7 @@ std::vector<minifi::core::Processor*>::iterator TestPlan::getProcessorItByUuid(c
std::shared_ptr<minifi::core::ProcessContext> TestPlan::getProcessContextForProcessor(minifi::core::Processor* processor) {
const auto contextMatchesProcessor = [&processor] (const std::shared_ptr<minifi::core::ProcessContext>& context) {
- return context->getProcessor().getUUIDStr() == processor->getUUIDStr();
+ return context->getProcessorInfo().getUUID().to_string() == processor->getUUIDStr();
};
const auto context_found_at = std::find_if(processor_contexts_.begin(), processor_contexts_.end(), contextMatchesProcessor);
if (context_found_at == processor_contexts_.end()) {
diff --git a/minifi-api/include/minifi-cpp/core/ClassLoader.h b/minifi-api/include/minifi-cpp/core/ClassLoader.h
index d1fcac4a0..cac9410d6 100644
--- a/minifi-api/include/minifi-cpp/core/ClassLoader.h
+++ b/minifi-api/include/minifi-cpp/core/ClassLoader.h
@@ -26,6 +26,7 @@
#include "Core.h"
#include "ObjectFactory.h"
+#include "ProcessorFactory.h"
namespace org::apache::nifi::minifi::core {
@@ -62,6 +63,8 @@ class ClassLoader {
*/
virtual void registerClass(const std::string &clazz, std::unique_ptr<ObjectFactory> factory) = 0;
+ virtual void registerClass(const std::string &clazz, std::unique_ptr<ProcessorFactory> factory) = 0;
+
virtual void unregisterClass(const std::string& clazz) = 0;
virtual std::optional<std::string> getGroupForClass(const std::string &class_name) const = 0;
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContext.h b/minifi-api/include/minifi-cpp/core/ProcessContext.h
index 6dc87a68e..0d319dcf9 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContext.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContext.h
@@ -33,8 +33,27 @@
namespace org::apache::nifi::minifi::core {
+namespace detail {
+template<typename T>
+concept NotAFlowFile = !std::convertible_to<T &, const FlowFile &> && !std::convertible_to<T &, const std::shared_ptr<FlowFile> &>;
+} // namespace detail
+
+class ProcessorInfo {
+ public:
+ virtual std::string getName() const = 0;
+ virtual utils::Identifier getUUID() const = 0;
+ virtual std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const = 0;
+ virtual std::map<std::string, core::Property> getSupportedProperties() const = 0;
+ virtual nonstd::expected<PropertyReference, std::error_code> getPropertyReference(std::string_view name) const = 0;
+
+ virtual ~ProcessorInfo() = default;
+};
+
+class Processor;
+
class ProcessContext : public virtual core::VariableRegistry, public virtual utils::EnableSharedFromThis {
public:
+ virtual const ProcessorInfo& getProcessorInfo() const = 0;
virtual Processor& getProcessor() const = 0;
virtual nonstd::expected<std::string, std::error_code> getProperty(std::string_view name, const FlowFile* flow_file = nullptr) const = 0;
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContextBuilder.h b/minifi-api/include/minifi-cpp/core/ProcessContextBuilder.h
index da20c1303..18d8ec27f 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContextBuilder.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContextBuilder.h
@@ -34,6 +34,8 @@ namespace org::apache::nifi::minifi::core {
* While this incurs a tiny cost to look up, it allows us to have a replaceable builder that erases the type we are
* constructing.
*/
+class Processor;
+
class ProcessContextBuilder : public virtual core::CoreComponent, public virtual utils::EnableSharedFromThis {
public:
virtual std::shared_ptr<ProcessContextBuilder> withProvider(core::controller::ControllerServiceProvider* controller_service_provider) = 0;
diff --git a/minifi-api/include/minifi-cpp/core/Processor.h b/minifi-api/include/minifi-cpp/core/Processor.h
index ded0f7570..e9cfda446 100644
--- a/minifi-api/include/minifi-cpp/core/Processor.h
+++ b/minifi-api/include/minifi-cpp/core/Processor.h
@@ -44,61 +44,30 @@ namespace core {
class ProcessContext;
class ProcessSession;
class ProcessSessionFactory;
+class ProcessorDescriptor;
-class Processor : public virtual Connectable, public virtual ConfigurableComponent, public virtual state::response::ResponseNodeSource {
+class ProcessorApi {
public:
- ~Processor() override = default;
+ virtual ~ProcessorApi() = default;
- virtual void setScheduledState(ScheduledState state) = 0;
- virtual ScheduledState getScheduledState() const = 0;
- virtual void setSchedulingStrategy(SchedulingStrategy strategy) = 0;
- virtual SchedulingStrategy getSchedulingStrategy() const = 0;
- virtual void setSchedulingPeriod(std::chrono::steady_clock::duration period) = 0;
- virtual std::chrono::steady_clock::duration getSchedulingPeriod() const = 0;
- virtual void setCronPeriod(const std::string &period) = 0;
- virtual std::string getCronPeriod() const = 0;
- virtual void setRunDurationNano(std::chrono::steady_clock::duration period) = 0;
- virtual std::chrono::steady_clock::duration getRunDurationNano() const = 0;
- virtual void setYieldPeriodMsec(std::chrono::milliseconds period) = 0;
- virtual std::chrono::steady_clock::duration getYieldPeriod() const = 0;
- virtual void setPenalizationPeriod(std::chrono::milliseconds period) = 0;
+ virtual bool isWorkAvailable() = 0;
+
+ virtual void restore(const std::shared_ptr<FlowFile>& file) = 0;
+
+
+ [[nodiscard]] virtual bool supportsDynamicProperties() const = 0;
+ [[nodiscard]] virtual bool supportsDynamicRelationships() const = 0;
+
+ virtual void initialize(ProcessorDescriptor& descriptor) = 0;
virtual bool isSingleThreaded() const = 0;
virtual std::string getProcessorType() const = 0;
- virtual void setTriggerWhenEmpty(bool value) = 0;
virtual bool getTriggerWhenEmpty() const = 0;
- virtual uint8_t getActiveTasks() const = 0;
- virtual void incrementActiveTasks() = 0;
- virtual void decrementActiveTask() = 0;
- virtual void clearActiveTask() = 0;
- using Connectable::yield;
- virtual void yield(std::chrono::steady_clock::duration delta_time) = 0;
- virtual bool isYield() = 0;
- virtual void clearYield() = 0;
- virtual std::chrono::steady_clock::time_point getYieldExpirationTime() const = 0;
- virtual std::chrono::steady_clock::duration getYieldTime() const = 0;
- virtual bool flowFilesOutGoingFull() const = 0;
- virtual bool addConnection(Connectable* connection) = 0;
- virtual void triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) = 0;
- virtual void trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) = 0;
virtual void onTrigger(ProcessContext&, ProcessSession&) = 0;
virtual void onSchedule(ProcessContext&, ProcessSessionFactory&) = 0;
virtual void onUnSchedule() = 0;
- virtual bool isThrottledByBackpressure() const = 0;
- virtual void validateAnnotations() const = 0;
+ virtual void notifyStop() = 0;
virtual annotation::Input getInputRequirement() const = 0;
virtual gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const = 0;
- virtual std::string getProcessGroupUUIDStr() const = 0;
- virtual void setProcessGroupUUIDStr(const std::string &uuid) = 0;
- virtual std::string getProcessGroupName() const = 0;
- virtual void setProcessGroupName(const std::string &name) = 0;
- virtual std::string getProcessGroupPath() const = 0;
- virtual void setProcessGroupPath(const std::string &path) = 0;
- virtual logging::LOG_LEVEL getLogBulletinLevel() const = 0;
- virtual void setLogBulletinLevel(logging::LOG_LEVEL level) = 0;
- virtual void setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback) = 0;
-
- virtual void updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force = false) = 0;
- virtual const std::unordered_map<Connection*, std::unordered_set<Processor*>>& reachable_processors() const = 0;
};
} // namespace core
diff --git a/minifi-api/include/minifi-cpp/core/ProcessorDescriptor.h b/minifi-api/include/minifi-cpp/core/ProcessorDescriptor.h
new file mode 100644
index 000000000..fd942605c
--- /dev/null
+++ b/minifi-api/include/minifi-cpp/core/ProcessorDescriptor.h
@@ -0,0 +1,33 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include "minifi-cpp/core/RelationshipDefinition.h"
+#include "minifi-cpp/core/PropertyDefinition.h"
+#include <span>
+
+namespace org::apache::nifi::minifi::core {
+
+class ProcessorDescriptor {
+ public:
+ virtual ~ProcessorDescriptor() = default;
+
+ virtual void setSupportedRelationships(std::span<const RelationshipDefinition> relationships) = 0;
+ virtual void setSupportedProperties(std::span<const PropertyReference> properties) = 0;
+};
+
+} // namespace org::apache::nifi::minifi::core
diff --git a/minifi-api/include/minifi-cpp/core/ProcessorFactory.h b/minifi-api/include/minifi-cpp/core/ProcessorFactory.h
new file mode 100644
index 000000000..7b453170f
--- /dev/null
+++ b/minifi-api/include/minifi-cpp/core/ProcessorFactory.h
@@ -0,0 +1,37 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include <string>
+#include <memory>
+#include <utility>
+#include "minifi-cpp/core/Processor.h"
+#include "minifi-cpp/core/ProcessorMetadata.h"
+
+namespace org::apache::nifi::minifi::core {
+
+class ProcessorFactory {
+ public:
+ virtual std::unique_ptr<ProcessorApi> create(ProcessorMetadata info) = 0;
+ virtual std::string getGroupName() const = 0;
+ virtual std::string getClassName() const = 0;
+
+ virtual ~ProcessorFactory() = default;
+};
+
+} // namespace org::apache::nifi::minifi::core
diff --git a/minifi-api/include/minifi-cpp/core/ProcessorMetadata.h b/minifi-api/include/minifi-cpp/core/ProcessorMetadata.h
new file mode 100644
index 000000000..b40e51faf
--- /dev/null
+++ b/minifi-api/include/minifi-cpp/core/ProcessorMetadata.h
@@ -0,0 +1,32 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include "minifi-cpp/utils/Id.h"
+#include "minifi-cpp/core/logging/Logger.h"
+#include <memory>
+#include <string>
+
+namespace org::apache::nifi::minifi::core {
+
+struct ProcessorMetadata {
+ utils::Identifier uuid;
+ std::string name;
+ std::shared_ptr<logging::Logger> logger;
+};
+
+} // namespace org::apache::nifi::minifi::core
diff --git a/minifi-api/include/minifi-cpp/core/state/nodes/MetricsBase.h b/minifi-api/include/minifi-cpp/core/state/nodes/MetricsBase.h
index 08fccb249..d8d390855 100644
--- a/minifi-api/include/minifi-cpp/core/state/nodes/MetricsBase.h
+++ b/minifi-api/include/minifi-cpp/core/state/nodes/MetricsBase.h
@@ -26,14 +26,14 @@
#include "../Value.h"
#include "../PublishedMetricProvider.h"
#include "core/Core.h"
-#include "core/Connectable.h"
+#include "minifi-cpp/core/Connectable.h"
namespace org::apache::nifi::minifi::state::response {
class ResponseNode;
using SharedResponseNode = gsl::not_null<std::shared_ptr<ResponseNode>>;
-class ResponseNode : public virtual core::Connectable, public virtual PublishedMetricProvider {
+class ResponseNode : public virtual core::CoreComponent, public virtual PublishedMetricProvider {
public:
~ResponseNode() override = default;
diff --git a/utils/include/core/ProcessContext.h b/utils/include/core/ProcessContext.h
index 2606058b3..4728236d6 100644
--- a/utils/include/core/ProcessContext.h
+++ b/utils/include/core/ProcessContext.h
@@ -46,51 +46,35 @@
namespace org::apache::nifi::minifi::core {
+class Processor;
+
class ProcessContextImpl : public core::VariableRegistryImpl, public virtual ProcessContext {
public:
ProcessContextImpl(Processor& processor, controller::ControllerServiceProvider* controller_service_provider, const std::shared_ptr<core::Repository>& repo,
- const std::shared_ptr<core::Repository>& flow_repo, const std::shared_ptr<core::ContentRepository>& content_repo = repository::createFileSystemRepository())
- : VariableRegistryImpl(static_cast<std::shared_ptr<Configure>>(minifi::Configure::create())),
- logger_(logging::LoggerFactory<ProcessContext>::getLogger()),
- controller_service_provider_(controller_service_provider),
- state_storage_(getStateStorage(logger_, controller_service_provider_, nullptr)),
- repo_(repo),
- flow_repo_(flow_repo),
- content_repo_(content_repo),
- processor_(processor),
- configure_(minifi::Configure::create()),
- initialized_(false) {}
+ const std::shared_ptr<core::Repository>& flow_repo, const std::shared_ptr<core::ContentRepository>& content_repo = repository::createFileSystemRepository());
ProcessContextImpl(Processor& processor, controller::ControllerServiceProvider* controller_service_provider, const std::shared_ptr<core::Repository>& repo,
const std::shared_ptr<core::Repository>& flow_repo, const std::shared_ptr<minifi::Configure>& configuration,
- const std::shared_ptr<core::ContentRepository>& content_repo = repository::createFileSystemRepository())
- : VariableRegistryImpl(configuration),
- logger_(logging::LoggerFactory<ProcessContext>::getLogger()),
- controller_service_provider_(controller_service_provider),
- state_storage_(getStateStorage(logger_, controller_service_provider_, configuration)),
- repo_(repo),
- flow_repo_(flow_repo),
- content_repo_(content_repo),
- processor_(processor),
- configure_(configuration ? gsl::make_not_null(configuration) : minifi::Configure::create()),
- initialized_(false) {}
+ const std::shared_ptr<core::ContentRepository>& content_repo = repository::createFileSystemRepository());
// Get Processor associated with the Process Context
Processor& getProcessor() const override { return processor_; }
+ const ProcessorInfo& getProcessorInfo() const override { return *info_; }
+
nonstd::expected<std::string, std::error_code> getProperty(std::string_view name, const FlowFile*) const override;
nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value) override;
nonstd::expected<void, std::error_code> clearProperty(std::string_view name) override;
nonstd::expected<std::string, std::error_code> getDynamicProperty(std::string_view name, const FlowFile*) const override;
nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value) override;
- std::vector<std::string> getDynamicPropertyKeys() const override { return processor_.getDynamicPropertyKeys(); }
- std::map<std::string, std::string> getDynamicProperties(const FlowFile*) const override { return processor_.getDynamicProperties(); }
+ std::vector<std::string> getDynamicPropertyKeys() const override;
+ std::map<std::string, std::string> getDynamicProperties(const FlowFile*) const override;
- bool isAutoTerminated(Relationship relationship) const override { return processor_.isAutoTerminated(relationship); }
- uint8_t getMaxConcurrentTasks() const override { return processor_.getMaxConcurrentTasks(); }
+ bool isAutoTerminated(Relationship relationship) const override;
+ uint8_t getMaxConcurrentTasks() const override;
- void yield() override { processor_.yield(); }
+ void yield() override;
std::shared_ptr<core::Repository> getProvenanceRepository() override { return repo_; }
@@ -127,11 +111,7 @@ class ProcessContextImpl : public core::VariableRegistryImpl, public virtual Pro
static constexpr char const* DefaultStateStorageName = "defaultstatestorage";
- StateManager* getStateManager() override {
- if (state_storage_ == nullptr) { return nullptr; }
- if (!state_manager_) { state_manager_ = state_storage_->getStateManager(processor_); }
- return state_manager_.get();
- }
+ StateManager* getStateManager() override;
bool hasStateManager() const override { return state_manager_ != nullptr; }
@@ -223,27 +203,8 @@ class ProcessContextImpl : public core::VariableRegistryImpl, public virtual Pro
std::shared_ptr<core::ContentRepository> content_repo_;
Processor& processor_;
gsl::not_null<std::shared_ptr<Configure>> configure_;
+ std::unique_ptr<ProcessorInfo> info_;
bool initialized_;
};
-inline nonstd::expected<std::string, std::error_code> ProcessContextImpl::getProperty(const std::string_view name, const FlowFile* const) const {
- return getProcessor().getProperty(name);
-}
-
-inline nonstd::expected<void, std::error_code> ProcessContextImpl::setProperty(const std::string_view name, std::string value) {
- return getProcessor().setProperty(name, std::move(value));
-}
-
-inline nonstd::expected<void, std::error_code> ProcessContextImpl::clearProperty(const std::string_view name) {
- return getProcessor().clearProperty(name);
-}
-
-inline nonstd::expected<std::string, std::error_code> ProcessContextImpl::getDynamicProperty(const std::string_view name, const FlowFile* const) const {
- return getProcessor().getDynamicProperty(name);
-}
-
-inline nonstd::expected<void, std::error_code> ProcessContextImpl::setDynamicProperty(std::string name, std::string value) {
- return getProcessor().setDynamicProperty(std::move(name), std::move(value));
-}
-
} // namespace org::apache::nifi::minifi::core
diff --git a/utils/include/core/Processor.h b/utils/include/core/Processor.h
index 11c779b00..3792098b7 100644
--- a/utils/include/core/Processor.h
+++ b/utils/include/core/Processor.h
@@ -16,290 +16,4 @@
*/
#pragma once
-#include <algorithm>
-#include <atomic>
-#include <chrono>
-#include <condition_variable>
-#include <functional>
-#include <memory>
-#include <mutex>
-#include <string>
-#include <string_view>
-#include <unordered_set>
-#include <unordered_map>
-#include <utility>
-#include <vector>
-
-#include "core/ConfigurableComponentImpl.h"
-#include "core/Connectable.h"
-#include "core/Property.h"
-#include "core/Core.h"
-#include "minifi-cpp/core/Annotation.h"
-#include "minifi-cpp/core/DynamicProperty.h"
-#include "minifi-cpp/core/Scheduling.h"
-#include "minifi-cpp/core/state/nodes/MetricsBase.h"
-#include "minifi-cpp/core/ProcessorMetrics.h"
-#include "utils/gsl.h"
-#include "utils/Id.h"
-#include "minifi-cpp/core/OutputAttributeDefinition.h"
-#include "minifi-cpp/core/Processor.h"
-
-#define ADD_GET_PROCESSOR_NAME \
- std::string getProcessorType() const override { \
- auto class_name = org::apache::nifi::minifi::core::className<decltype(*this)>(); \
- auto splitted = org::apache::nifi::minifi::utils::string::split(class_name, "::"); \
- return splitted[splitted.size() - 1]; \
- }
-
-#define ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS \
- bool supportsDynamicProperties() const override { return SupportsDynamicProperties; } \
- bool supportsDynamicRelationships() const override { return SupportsDynamicRelationships; } \
- minifi::core::annotation::Input getInputRequirement() const override { return InputRequirement; } \
- bool isSingleThreaded() const override { return IsSingleThreaded; } \
- ADD_GET_PROCESSOR_NAME
-
-namespace org::apache::nifi::minifi {
-
-class Connection;
-
-namespace core {
-
-class ProcessContext;
-class ProcessSession;
-class ProcessSessionFactory;
-
-constexpr std::chrono::microseconds MINIMUM_SCHEDULING_PERIOD{30};
-
-#define BUILDING_DLL 1
-
-class ProcessorImpl : public virtual Processor, public ConnectableImpl, public ConfigurableComponentImpl {
- public:
- ProcessorImpl(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<ProcessorMetrics> metrics = nullptr);
- explicit ProcessorImpl(std::string_view name, std::shared_ptr<ProcessorMetrics> metrics = nullptr);
-
- ProcessorImpl(const ProcessorImpl& parent) = delete;
- ProcessorImpl& operator=(const ProcessorImpl& parent) = delete;
-
- bool isRunning() const override;
-
- ~ProcessorImpl() override;
-
- void setScheduledState(ScheduledState state) override;
-
- ScheduledState getScheduledState() const override {
- return state_;
- }
-
- void setSchedulingStrategy(SchedulingStrategy strategy) override {
- strategy_ = strategy;
- }
-
- SchedulingStrategy getSchedulingStrategy() const override {
- return strategy_;
- }
-
- void setSchedulingPeriod(std::chrono::steady_clock::duration period) override {
- scheduling_period_ = std::max(std::chrono::steady_clock::duration(MINIMUM_SCHEDULING_PERIOD), period);
- }
-
- std::chrono::steady_clock::duration getSchedulingPeriod() const override {
- return scheduling_period_;
- }
-
- void setCronPeriod(const std::string &period) override {
- cron_period_ = period;
- }
-
- std::string getCronPeriod() const override {
- return cron_period_;
- }
-
- void setRunDurationNano(std::chrono::steady_clock::duration period) override {
- run_duration_ = period;
- }
-
- std::chrono::steady_clock::duration getRunDurationNano() const override {
- return (run_duration_);
- }
-
- void setYieldPeriodMsec(std::chrono::milliseconds period) override {
- yield_period_ = period;
- }
-
- std::chrono::steady_clock::duration getYieldPeriod() const override {
- return yield_period_;
- }
-
- void setPenalizationPeriod(std::chrono::milliseconds period) override {
- penalization_period_ = period;
- }
-
- void setMaxConcurrentTasks(uint8_t tasks) override;
-
- bool isSingleThreaded() const override = 0;
-
- std::string getProcessorType() const override = 0;
-
- void setTriggerWhenEmpty(bool value) override {
- _triggerWhenEmpty = value;
- }
-
- bool getTriggerWhenEmpty() const override {
- return (_triggerWhenEmpty);
- }
-
- uint8_t getActiveTasks() const override {
- return (active_tasks_);
- }
-
- void incrementActiveTasks() override {
- ++active_tasks_;
- }
-
- void decrementActiveTask() override {
- if (active_tasks_ > 0)
- --active_tasks_;
- }
-
- void clearActiveTask() override {
- active_tasks_ = 0;
- }
-
- std::string getProcessGroupUUIDStr() const override {
- return process_group_uuid_;
- }
-
- void setProcessGroupUUIDStr(const std::string &uuid) override {
- process_group_uuid_ = uuid;
- }
-
- std::string getProcessGroupName() const override {
- return process_group_name_;
- }
-
- void setProcessGroupName(const std::string &name) override {
- process_group_name_ = name;
- }
-
- std::string getProcessGroupPath() const override {
- return process_group_path_;
- }
-
- void setProcessGroupPath(const std::string &path) override {
- process_group_path_ = path;
- }
-
- void yield() override;
-
- void yield(std::chrono::steady_clock::duration delta_time) override;
-
- bool isYield() override;
-
- void clearYield() override;
-
- std::chrono::steady_clock::time_point getYieldExpirationTime() const override { return yield_expiration_; }
- std::chrono::steady_clock::duration getYieldTime() const override;
- // Whether flow file queue full in any of the outgoing connection
- bool flowFilesOutGoingFull() const override;
-
- bool addConnection(Connectable* connection) override;
-
- bool canEdit() override {
- return !isRunning();
- }
-
- void initialize() override {
- }
-
- void triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) override;
- void trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) override;
-
- void onTrigger(ProcessContext&, ProcessSession&) override {}
-
- void onSchedule(ProcessContext&, ProcessSessionFactory&) override {}
-
- // Hook executed when onSchedule fails (throws). Configuration should be reset in this
- void onUnSchedule() override {
- notifyStop();
- }
-
- // Check all incoming connections for work
- bool isWorkAvailable() override;
-
- bool isThrottledByBackpressure() const override;
-
- Connectable* pickIncomingConnection() override;
-
- void validateAnnotations() const override;
-
- annotation::Input getInputRequirement() const override = 0;
-
- state::response::SharedResponseNode getResponseNode() override {
- return metrics_;
- }
-
- gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const override {
- return metrics_;
- }
-
- logging::LOG_LEVEL getLogBulletinLevel() const override {
- return log_bulletin_level_;
- }
-
- void setLogBulletinLevel(logging::LOG_LEVEL level) override {
- log_bulletin_level_ = level;
- }
-
- void setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback) override;
-
- static constexpr auto DynamicProperties = std::array<DynamicProperty, 0>{};
-
- static constexpr auto OutputAttributes = std::array<OutputAttributeReference, 0>{};
-
- protected:
- virtual void notifyStop() {
- }
-
- std::atomic<ScheduledState> state_;
-
- std::atomic<std::chrono::steady_clock::duration> scheduling_period_;
- std::atomic<std::chrono::steady_clock::duration> run_duration_;
- std::atomic<std::chrono::steady_clock::duration> yield_period_;
-
- std::atomic<uint8_t> active_tasks_;
- std::atomic<bool> _triggerWhenEmpty;
-
- std::string cron_period_;
- gsl::not_null<std::shared_ptr<ProcessorMetrics>> metrics_;
-
- std::shared_ptr<logging::Logger> logger_;
- logging::LOG_LEVEL log_bulletin_level_ = logging::LOG_LEVEL::warn;
-
- private:
- mutable std::mutex mutex_;
- std::atomic<std::chrono::steady_clock::time_point> yield_expiration_{};
-
- static std::mutex& getGraphMutex() {
- static std::mutex mutex{};
- return mutex;
- }
-
- // must hold the graphMutex
- void updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force = false) override;
-
- const std::unordered_map<Connection*, std::unordered_set<Processor*>>& reachable_processors() const override {
- return reachable_processors_;
- }
-
- static bool partOfCycle(Connection* conn);
-
- // an outgoing connection allows us to reach these nodes
- std::unordered_map<Connection*, std::unordered_set<Processor*>> reachable_processors_;
-
- std::string process_group_uuid_;
- std::string process_group_name_;
- std::string process_group_path_;
-};
-
-} // namespace core
-} // namespace org::apache::nifi::minifi
+#include "core/ProcessorImpl.h"
diff --git a/utils/include/core/ProcessorFactory.h b/utils/include/core/ProcessorFactory.h
new file mode 100644
index 000000000..17a42b78d
--- /dev/null
+++ b/utils/include/core/ProcessorFactory.h
@@ -0,0 +1,57 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include <string>
+#include <memory>
+#include <utility>
+#include "ClassName.h"
+#include "minifi-cpp/core/ProcessorFactory.h"
+
+namespace org::apache::nifi::minifi::core {
+
+template<class T>
+class ProcessorFactoryImpl : public ProcessorFactory {
+ public:
+ ProcessorFactoryImpl()
+ : class_name_(core::className<T>()) {
+ }
+
+ explicit ProcessorFactoryImpl(std::string group_name)
+ : group_name_(std::move(group_name)),
+ class_name_(core::className<T>()) {
+ }
+
+ std::string getGroupName() const override {
+ return group_name_;
+ }
+
+ std::unique_ptr<ProcessorApi> create(ProcessorMetadata info) override {
+ return std::make_unique<T>(info);
+ }
+
+ std::string getClassName() const override {
+ return class_name_;
+ }
+
+ protected:
+ std::string group_name_;
+ std::string class_name_;
+};
+
+} // namespace org::apache::nifi::minifi::core
diff --git a/utils/include/core/ProcessorImpl.h b/utils/include/core/ProcessorImpl.h
new file mode 100644
index 000000000..633a80c70
--- /dev/null
+++ b/utils/include/core/ProcessorImpl.h
@@ -0,0 +1,159 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include <algorithm>
+#include <atomic>
+#include <chrono>
+#include <condition_variable>
+#include <functional>
+#include <memory>
+#include <mutex>
+#include <string>
+#include <string_view>
+#include <unordered_set>
+#include <unordered_map>
+#include <utility>
+#include <vector>
+
+#include "core/ConfigurableComponentImpl.h"
+#include "core/Connectable.h"
+#include "core/Property.h"
+#include "core/Core.h"
+#include "minifi-cpp/core/Annotation.h"
+#include "minifi-cpp/core/DynamicProperty.h"
+#include "minifi-cpp/core/Scheduling.h"
+#include "minifi-cpp/core/state/nodes/MetricsBase.h"
+#include "minifi-cpp/core/ProcessorMetrics.h"
+#include "utils/gsl.h"
+#include "utils/Id.h"
+#include "minifi-cpp/core/OutputAttributeDefinition.h"
+#include "minifi-cpp/core/Processor.h"
+#include "minifi-cpp/utils/PropertyErrors.h"
+#include "minifi-cpp/core/ProcessorMetadata.h"
+#include "Exception.h"
+
+#define ADD_GET_PROCESSOR_NAME \
+ std::string getProcessorType() const override { \
+ auto class_name = org::apache::nifi::minifi::core::className<decltype(*this)>(); \
+ auto splitted = org::apache::nifi::minifi::utils::string::split(class_name, "::"); \
+ return splitted[splitted.size() - 1]; \
+ }
+
+#define ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS \
+ bool supportsDynamicProperties() const override { return SupportsDynamicProperties; } \
+ bool supportsDynamicRelationships() const override { return SupportsDynamicRelationships; } \
+ minifi::core::annotation::Input getInputRequirement() const override { return InputRequirement; } \
+ bool isSingleThreaded() const override { return IsSingleThreaded; } \
+ ADD_GET_PROCESSOR_NAME
+
+namespace org::apache::nifi::minifi {
+
+class Connection;
+
+namespace core {
+
+class ProcessContext;
+class ProcessSession;
+class ProcessSessionFactory;
+
+constexpr std::chrono::microseconds MINIMUM_SCHEDULING_PERIOD{30};
+
+#define BUILDING_DLL 1
+
+class ProcessorImpl : public virtual ProcessorApi {
+ public:
+ explicit ProcessorImpl(ProcessorMetadata info, std::shared_ptr<ProcessorMetrics> metrics = nullptr);
+ explicit ProcessorImpl(std::string_view name, const utils::Identifier& uuid = utils::IdGenerator::getIdGenerator()->generate(), std::shared_ptr<core::logging::Logger> logger = nullptr, std::shared_ptr<ProcessorMetrics> metrics = nullptr);
+
+ ProcessorImpl(const ProcessorImpl& parent) = delete;
+ ProcessorImpl& operator=(const ProcessorImpl& parent) = delete;
+
+ ~ProcessorImpl() override;
+
+ bool isSingleThreaded() const override = 0;
+
+ [[nodiscard]] bool supportsDynamicProperties() const override = 0;
+
+ [[nodiscard]] bool supportsDynamicRelationships() const override = 0;
+
+ std::string getProcessorType() const override = 0;
+
+ void setTriggerWhenEmpty(bool trigger_when_empty) {
+ trigger_when_empty_ = trigger_when_empty;
+ }
+
+ bool getTriggerWhenEmpty() const override {
+ return trigger_when_empty_;
+ }
+
+ void initialize(ProcessorDescriptor& self) final;
+
+ void setSupportedRelationships(std::span<const RelationshipDefinition> relationships);
+
+ void setSupportedProperties(std::span<const PropertyReference> properties);
+
+ virtual void initialize() {}
+
+ void onTrigger(ProcessContext&, ProcessSession&) override {}
+
+ void onSchedule(ProcessContext&, ProcessSessionFactory&) override {}
+
+ // Hook executed when onSchedule fails (throws). Configuration should be reset in this
+ void onUnSchedule() override {
+ notifyStop();
+ }
+
+ // Check all incoming connections for work
+ bool isWorkAvailable() override;
+
+ annotation::Input getInputRequirement() const override = 0;
+
+ gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const override {
+ return metrics_;
+ }
+
+ static constexpr auto DynamicProperties = std::array<DynamicProperty, 0>{};
+
+ static constexpr auto OutputAttributes = std::array<OutputAttributeReference, 0>{};
+
+ void restore(const std::shared_ptr<FlowFile>& file) override;
+
+ std::string getName() const;
+ utils::Identifier getUUID() const;
+ utils::SmallString<36> getUUIDStr() const;
+
+ protected:
+ void notifyStop() override {
+ }
+
+ ProcessorMetadata info_;
+
+ std::atomic<bool> trigger_when_empty_;
+
+ gsl::not_null<std::shared_ptr<ProcessorMetrics>> metrics_;
+
+ std::shared_ptr<logging::Logger> logger_;
+
+ private:
+ mutable std::mutex mutex_;
+
+ ProcessorDescriptor* descriptor_{nullptr};
+};
+
+} // namespace core
+} // namespace org::apache::nifi::minifi
diff --git a/utils/include/core/ProcessorMetrics.h b/utils/include/core/ProcessorMetrics.h
index 5de882d8c..40e0e6e0a 100644
--- a/utils/include/core/ProcessorMetrics.h
+++ b/utils/include/core/ProcessorMetrics.h
@@ -35,11 +35,11 @@ concept Summable = requires(T x) { x + x; }; // NOLINT(readability/braces)
template<typename T>
concept DividableByInteger = requires(T x, uint32_t divisor) { x / divisor; }; // NOLINT(readability/braces)
-class Processor;
+class ProcessorImpl;
class ProcessorMetricsImpl : public state::response::ResponseNodeImpl, public virtual ProcessorMetrics {
public:
- explicit ProcessorMetricsImpl(const Processor& source_processor);
+ explicit ProcessorMetricsImpl(const ProcessorImpl& source_processor);
[[nodiscard]] std::string getName() const override;
@@ -97,7 +97,7 @@ class ProcessorMetricsImpl : public state::response::ResponseNodeImpl, public vi
mutable std::mutex transferred_relationships_mutex_;
std::unordered_map<std::string, size_t> transferred_relationships_;
- const Processor& source_processor_;
+ const ProcessorImpl& source_processor_;
Averager<std::chrono::milliseconds> on_trigger_runtime_averager_;
Averager<std::chrono::milliseconds> session_commit_runtime_averager_;
diff --git a/utils/include/core/Resource.h b/utils/include/core/Resource.h
index ab4527b25..c4994666a 100644
--- a/utils/include/core/Resource.h
+++ b/utils/include/core/Resource.h
@@ -29,6 +29,7 @@
#include "agent/agent_docs.h"
#include "utils/OptionalUtils.h"
#include "utils/Macro.h"
+#include "core/ProcessorFactory.h"
namespace org::apache::nifi::minifi::core {
@@ -54,9 +55,16 @@ class StaticClassType {
auto module_name = "minifi-system";
#endif
- for (const auto& construction_name : construction_names_) {
- auto factory = std::unique_ptr<ObjectFactory>(new DefaultObjectFactory<Class>(module_name));
- getClassLoader().registerClass(construction_name, std::move(factory));
+ if constexpr (Type == ResourceType::Processor) {
+ for (const auto& construction_name : construction_names_) {
+ auto factory = std::unique_ptr<ProcessorFactory>(new ProcessorFactoryImpl<Class>(module_name));
+ getClassLoader().registerClass(construction_name, std::move(factory));
+ }
+ } else {
+ for (const auto& construction_name : construction_names_) {
+ auto factory = std::unique_ptr<ObjectFactory>(new DefaultObjectFactory<Class>(module_name));
+ getClassLoader().registerClass(construction_name, std::move(factory));
+ }
}
minifi::AgentDocs::createClassDescription<Class, Type>(module_name, class_name);
diff --git a/utils/include/core/state/nodes/ResponseNode.h b/utils/include/core/state/nodes/ResponseNode.h
index 2bbb2f50b..2500998dc 100644
--- a/utils/include/core/state/nodes/ResponseNode.h
+++ b/utils/include/core/state/nodes/ResponseNode.h
@@ -25,7 +25,6 @@
#include <unordered_map>
#include "core/Core.h"
-#include "core/Connectable.h"
#include "minifi-cpp/core/state/nodes/MetricsBase.h"
#include "core/state/PublishedMetricProvider.h"
@@ -37,20 +36,20 @@ using SharedResponseNode = gsl::not_null<std::shared_ptr<ResponseNode>>;
/**
* Purpose: Defines a metric. Serialization is intended to be thread safe.
*/
-class ResponseNodeImpl : public core::ConnectableImpl, public PublishedMetricProviderImpl, public virtual ResponseNode {
+class ResponseNodeImpl : public core::CoreComponentImpl, public PublishedMetricProviderImpl, public virtual ResponseNode {
public:
ResponseNodeImpl()
- : core::ConnectableImpl("metric"),
+ : core::CoreComponentImpl("metric"),
is_array_(false) {
}
explicit ResponseNodeImpl(const std::string_view name)
- : core::ConnectableImpl(name),
+ : core::CoreComponentImpl(name),
is_array_(false) {
}
ResponseNodeImpl(const std::string_view name, const utils::Identifier& uuid)
- : core::ConnectableImpl(name, uuid),
+ : core::CoreComponentImpl(name, uuid),
is_array_(false) {
}
@@ -58,17 +57,6 @@ class ResponseNodeImpl : public core::ConnectableImpl, public PublishedMetricPro
static std::vector<SerializedResponseNode> serializeAndMergeResponseNodes(const std::vector<SharedResponseNode>& nodes);
- void yield() override {
- }
-
- bool isRunning() const override {
- return true;
- }
-
- bool isWorkAvailable() override {
- return true;
- }
-
bool isArray() const override {
return is_array_;
}
diff --git a/utils/src/core/Processor.cpp b/utils/src/core/Processor.cpp
index 95feac9f8..e70957e21 100644
--- a/utils/src/core/Processor.cpp
+++ b/utils/src/core/Processor.cpp
@@ -38,335 +38,68 @@
#include "fmt/format.h"
#include "Exception.h"
#include "core/ProcessorMetrics.h"
+#include "minifi-cpp/utils/PropertyErrors.h"
+#include "minifi-cpp/core/ProcessorDescriptor.h"
using namespace std::literals::chrono_literals;
namespace org::apache::nifi::minifi::core {
-ProcessorImpl::ProcessorImpl(std::string_view name, std::shared_ptr<ProcessorMetrics> metrics)
- : ConnectableImpl(name),
- state_(DISABLED),
- scheduling_period_(MINIMUM_SCHEDULING_PERIOD),
- run_duration_(DEFAULT_RUN_DURATION),
- yield_period_(DEFAULT_YIELD_PERIOD_SECONDS),
- active_tasks_(0),
- _triggerWhenEmpty(false),
+ProcessorImpl::ProcessorImpl(ProcessorMetadata info, std::shared_ptr<ProcessorMetrics> metrics)
+ : info_(info),
+ trigger_when_empty_(false),
metrics_(metrics ? std::move(metrics) : std::make_shared<ProcessorMetricsImpl>(*this)),
- logger_(logging::LoggerFactory<Processor>::getLogger(uuid_)) {
- has_work_.store(false);
- // Setup the default values
- strategy_ = TIMER_DRIVEN;
- penalization_period_ = DEFAULT_PENALIZATION_PERIOD;
- max_concurrent_tasks_ = DEFAULT_MAX_CONCURRENT_TASKS;
- incoming_connections_Iter = this->incoming_connections_.begin();
- logger_->log_debug("Processor {} created UUID {}", name_, getUUIDStr());
+ logger_(info.logger) {
+ logger_->log_debug("Processor {} created with uuid {}", getName(), getUUIDStr());
}
-ProcessorImpl::ProcessorImpl(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<ProcessorMetrics> metrics)
- : ConnectableImpl(name, uuid),
- state_(DISABLED),
- scheduling_period_(MINIMUM_SCHEDULING_PERIOD),
- run_duration_(DEFAULT_RUN_DURATION),
- yield_period_(DEFAULT_YIELD_PERIOD_SECONDS),
- active_tasks_(0),
- _triggerWhenEmpty(false),
- metrics_(metrics ? std::move(metrics) : std::make_shared<ProcessorMetricsImpl>(*this)),
- logger_(logging::LoggerFactory<Processor>::getLogger(uuid_)) {
- has_work_.store(false);
- // Setup the default values
- strategy_ = TIMER_DRIVEN;
- penalization_period_ = DEFAULT_PENALIZATION_PERIOD;
- max_concurrent_tasks_ = DEFAULT_MAX_CONCURRENT_TASKS;
- incoming_connections_Iter = this->incoming_connections_.begin();
- logger_->log_debug("Processor {} created with uuid {}", name_, getUUIDStr());
-}
+ProcessorImpl::ProcessorImpl(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger, std::shared_ptr<ProcessorMetrics> metrics)
+ : ProcessorImpl(ProcessorMetadata{
+ .uuid = uuid,
+ .name = std::string{name},
+ .logger = logger ? logger : logging::LoggerFactory<ProcessorImpl>::getLogger(uuid)
+ }, metrics)
+{}
ProcessorImpl::~ProcessorImpl() {
- logger_->log_debug("Destroying processor {} with uuid {}", name_, getUUIDStr());
-}
-
-bool ProcessorImpl::isRunning() const {
- return (state_ == RUNNING && active_tasks_ > 0);
-}
-
-void ProcessorImpl::setScheduledState(ScheduledState state) {
- state_ = state;
- if (state == STOPPED) {
- notifyStop();
- }
-}
-
-bool ProcessorImpl::addConnection(Connectable* conn) {
- enum class SetAs{
- NONE,
- OUTPUT,
- INPUT,
- };
- SetAs result = SetAs::NONE;
-
- if (isRunning()) {
- logger_->log_warn("Can not add connection while the process {} is running", name_);
- return false;
- }
- const auto connection = dynamic_cast<Connection*>(conn);
- if (!connection) {
- return false;
- }
-
- std::lock_guard<std::mutex> lock(getGraphMutex());
-
- auto updateGraph = gsl::finally([&] {
- if (result == SetAs::INPUT) {
- updateReachability(lock);
- } else if (result == SetAs::OUTPUT) {
- updateReachability(lock, true);
- }
- });
-
- utils::Identifier srcUUID = connection->getSourceUUID();
- utils::Identifier destUUID = connection->getDestinationUUID();
-
- if (uuid_ == destUUID) {
- // Connection is destination to the current processor
- if (!incoming_connections_.contains(connection)) {
- incoming_connections_.insert(connection);
- connection->setDestination(this);
- logger_->log_debug("Add connection {} into Processor {} incoming connection", connection->getName(), name_);
- incoming_connections_Iter = this->incoming_connections_.begin();
- result = SetAs::OUTPUT;
- }
- }
- if (uuid_ == srcUUID) {
- for (const auto& rel : connection->getRelationships()) {
- const auto relationship = rel.getName();
- // Connection is source from the current processor
- auto &&it = outgoing_connections_.find(relationship);
- if (it != outgoing_connections_.end()) {
- // We already has connection for this relationship
- std::set<Connectable*> existedConnection = it->second;
- if (!existedConnection.contains(connection)) {
- // We do not have the same connection for this relationship yet
- existedConnection.insert(connection);
- connection->setSource(this);
- outgoing_connections_[relationship] = existedConnection;
- logger_->log_debug("Add connection {} into Processor {} outgoing connection for relationship {}", connection->getName(), name_, relationship);
- result = SetAs::INPUT;
- }
- } else {
- // We do not have any outgoing connection for this relationship yet
- std::set<Connectable*> newConnection;
- newConnection.insert(connection);
- connection->setSource(this);
- outgoing_connections_[relationship] = newConnection;
- logger_->log_debug("Add connection {} into Processor {} outgoing connection for relationship {}", connection->getName(), name_, relationship);
- result = SetAs::INPUT;
- }
- }
- }
- return result != SetAs::NONE;
-}
-
-bool ProcessorImpl::flowFilesOutGoingFull() const {
- std::lock_guard<std::mutex> lock(mutex_);
-
- for (const auto& [_name, existed_connection] : outgoing_connections_) {
- if (ranges::any_of(existed_connection, [](const Connectable* conn) {
- const auto connection = dynamic_cast<const Connection*>(conn);
- return connection && connection->backpressureThresholdReached();
- })) {
- return true;
- }
- }
-
- return false;
-}
-
-void ProcessorImpl::triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) {
- const auto process_session = session_factory->createSession();
- process_session->setMetrics(metrics_);
- try {
- trigger(context, process_session);
- process_session->commit();
- } catch (const std::exception& exception) {
- logger_->log_warn("Caught \"{}\" ({}) during Processor::onTrigger of processor: {} ({})",
- exception.what(), typeid(exception).name(), getUUIDStr(), getName());
- process_session->rollback();
- throw;
- } catch (...) {
- logger_->log_warn("Caught unknown exception during Processor::onTrigger of processor: {} ({})", getUUIDStr(), getName());
- process_session->rollback();
- throw;
- }
-}
-
-void ProcessorImpl::trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) {
- ++metrics_->invocations();
- const auto start = std::chrono::steady_clock::now();
- onTrigger(*context, *process_session);
- metrics_->addLastOnTriggerRuntime(std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - start));
+ logger_->log_debug("Destroying processor {} with uuid {}", getName(), getUUIDStr());
}
bool ProcessorImpl::isWorkAvailable() {
- // We have work if any incoming connection has work
- std::lock_guard<std::mutex> lock(mutex_);
- bool hasWork = false;
-
- try {
- for (const auto &conn : incoming_connections_) {
- auto connection = dynamic_cast<Connection*>(conn);
- if (!connection) {
- continue;
- }
- if (connection->isWorkAvailable()) {
- hasWork = true;
- break;
- }
- }
- } catch (...) {
- logger_->log_error("Caught an exception (type: {}) while checking if work is available;"
- " unless it was positively determined that work is available, assuming NO work is available!",
- getCurrentExceptionTypeName());
- }
-
- return hasWork;
-}
-
-// must hold the graphMutex
-void ProcessorImpl::updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force) {
- bool didChange = force;
- for (auto& outIt : outgoing_connections_) {
- for (auto& outConn : outIt.second) {
- auto connection = dynamic_cast<Connection*>(outConn);
- if (!connection) {
- continue;
- }
- auto dest = dynamic_cast<Processor*>(connection->getDestination());
- if (!dest) {
- continue;
- }
- if (reachable_processors_[connection].insert(dest).second) {
- didChange = true;
- }
- for (auto& reachedIt : dest->reachable_processors()) {
- for (auto &reached_proc : reachedIt.second) {
- if (reachable_processors_[connection].insert(reached_proc).second) {
- didChange = true;
- }
- }
- }
- }
- }
- if (didChange) {
- // propagate the change to sources
- for (auto& inConn : incoming_connections_) {
- auto connection = dynamic_cast<Connection*>(inConn);
- if (!connection) {
- continue;
- }
- auto source = dynamic_cast<Processor*>(connection->getSource());
- if (!source) {
- continue;
- }
- source->updateReachability(graph_lock);
- }
- }
-}
-
-bool ProcessorImpl::partOfCycle(Connection* conn) {
- auto source = dynamic_cast<Processor*>(conn->getSource());
- if (!source) {
- return false;
- }
- auto it = source->reachable_processors().find(conn);
- if (it == source->reachable_processors().end()) {
- return false;
- }
- return it->second.contains(source);
-}
-
-bool ProcessorImpl::isThrottledByBackpressure() const {
- bool isThrottledByOutgoing = ranges::any_of(outgoing_connections_, [](auto& name_connection_set_pair) {
- return ranges::any_of(name_connection_set_pair.second, [](auto& connectable) {
- auto connection = dynamic_cast<Connection*>(connectable);
- return connection && connection->backpressureThresholdReached();
- });
- });
- bool isForcedByIncomingCycle = ranges::any_of(incoming_connections_, [](auto& connectable) {
- auto connection = dynamic_cast<Connection*>(connectable);
- return connection && partOfCycle(connection) && connection->backpressureThresholdReached();
- });
- return isThrottledByOutgoing && !isForcedByIncomingCycle;
-}
-
-Connectable* ProcessorImpl::pickIncomingConnection() {
- std::lock_guard<std::mutex> rel_guard(relationship_mutex_);
-
- auto beginIt = incoming_connections_Iter;
- Connectable* inConn = nullptr;
- do {
- inConn = getNextIncomingConnectionImpl(rel_guard);
- auto connection = dynamic_cast<Connection*>(inConn);
- if (!connection) {
- continue;
- }
- if (partOfCycle(connection) && connection->backpressureThresholdReached()) {
- return inConn;
- }
- } while (incoming_connections_Iter != beginIt);
-
- // we did not find a preferred connection
- return getNextIncomingConnectionImpl(rel_guard);
+ return false;
}
-void ProcessorImpl::validateAnnotations() const {
- switch (getInputRequirement()) {
- case annotation::Input::INPUT_REQUIRED: {
- if (!hasIncomingConnections()) {
- throw Exception(PROCESS_SCHEDULE_EXCEPTION, fmt::format("INPUT_REQUIRED was specified for the processor '{}' (uuid: '{}'), but no incoming connections were found",
- getName(), std::string(getUUIDStr())));
- }
- break;
- }
- case annotation::Input::INPUT_ALLOWED:
- break;
- case annotation::Input::INPUT_FORBIDDEN: {
- if (hasIncomingConnections()) {
- throw Exception(PROCESS_SCHEDULE_EXCEPTION, fmt::format("INPUT_FORBIDDEN was specified for the processor '{}' (uuid: '{}'), but there are incoming connections",
- getName(), std::string(getUUIDStr())));
- }
- }
- }
+void ProcessorImpl::restore(const std::shared_ptr<FlowFile>& file) {
+ gsl_Assert("Not implemented");
}
-void ProcessorImpl::setMaxConcurrentTasks(const uint8_t tasks) {
- if (isSingleThreaded() && tasks > 1) {
- logger_->log_warn("Processor {} can not be run in parallel, its \"max concurrent tasks\" value is too high. "
- "It was set to 1 from {}.", name_, tasks);
- max_concurrent_tasks_ = 1;
- return;
- }
-
- max_concurrent_tasks_ = tasks;
+std::string ProcessorImpl::getName() const {
+ return info_.name;
}
-void ProcessorImpl::yield() {
- yield_expiration_ = std::chrono::steady_clock::now() + yield_period_.load();
+utils::Identifier ProcessorImpl::getUUID() const {
+ return info_.uuid;
}
-void ProcessorImpl::yield(std::chrono::steady_clock::duration delta_time) {
- yield_expiration_ = std::chrono::steady_clock::now() + delta_time;
+utils::SmallString<36> ProcessorImpl::getUUIDStr() const {
+ return getUUID().to_string();
}
-bool ProcessorImpl::isYield() {
- return getYieldTime() > 0ms;
+void ProcessorImpl::initialize(ProcessorDescriptor& self) {
+ gsl_Expects(!descriptor_);
+ descriptor_ = &self;
+ auto guard = gsl::finally([&] {descriptor_ = nullptr;});
+ initialize();
}
-void ProcessorImpl::clearYield() {
- yield_expiration_ = std::chrono::steady_clock::time_point();
+void ProcessorImpl::setSupportedRelationships(std::span<const RelationshipDefinition> relationships) {
+ gsl_Expects(descriptor_);
+ descriptor_->setSupportedRelationships(relationships);
}
-std::chrono::steady_clock::duration ProcessorImpl::getYieldTime() const {
- return std::max(yield_expiration_.load()-std::chrono::steady_clock::now(), std::chrono::steady_clock::duration{0});
+void ProcessorImpl::setSupportedProperties(std::span<const PropertyReference> properties) {
+ gsl_Expects(descriptor_);
+ descriptor_->setSupportedProperties(properties);
}
void ProcessorImpl::setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback) {
diff --git a/utils/src/core/ProcessorMetrics.cpp b/utils/src/core/ProcessorMetrics.cpp
index 713822540..5d3e7be33 100644
--- a/utils/src/core/ProcessorMetrics.cpp
+++ b/utils/src/core/ProcessorMetrics.cpp
@@ -25,7 +25,7 @@ using namespace std::literals::chrono_literals;
namespace org::apache::nifi::minifi::core {
-ProcessorMetricsImpl::ProcessorMetricsImpl(const Processor& source_processor)
+ProcessorMetricsImpl::ProcessorMetricsImpl(const ProcessorImpl& source_processor)
: source_processor_(source_processor),
on_trigger_runtime_averager_(STORED_ON_TRIGGER_RUNTIME_COUNT),
session_commit_runtime_averager_(STORED_ON_TRIGGER_RUNTIME_COUNT) {
(nifi-minifi-cpp) 21/37: Rename
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit c4cd764c12ad7ffdc43222eb52ffd0a4435247cc
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Jun 23 14:34:34 2025 +0200
Rename
---
CMakeLists.txt | 4 ++--
core-framework/CMakeLists.txt | 22 ++++++++++++++++++++++
{utils => core-framework}/include/Exception.h | 0
{utils => core-framework}/include/SwapManager.h | 0
.../include/agent/agent_docs.h | 0
.../include/agent/agent_version.h | 0
.../include/core/Annotation.h | 0
.../include/core/BufferedContentSession.h | 0
.../include/core/ClassLoader.h | 0
{utils => core-framework}/include/core/ClassName.h | 0
.../include/core/ConfigurableComponentImpl.h | 0
.../include/core/Connectable.h | 0
.../include/core/ContentRepository.h | 0
.../include/core/ContentSession.h | 0
{utils => core-framework}/include/core/Core.h | 0
.../include/core/ObjectFactory.h | 0
.../include/core/OutputAttributeDefinition.h | 0
.../include/core/ProcessContext.h | 0
.../include/core/ProcessContextBuilder.h | 0
.../include/core/ProcessorFactory.h | 0
.../include/core/ProcessorImpl.h | 0
.../include/core/ProcessorMetrics.h | 0
{utils => core-framework}/include/core/Property.h | 0
.../include/core/PropertyDefinition.h | 0
.../include/core/PropertyDefinitionBuilder.h | 0
.../include/core/Relationship.h | 0
.../include/core/RelationshipDefinition.h | 0
.../include/core/Repository.h | 0
.../include/core/RepositoryMetricsSource.h | 0
{utils => core-framework}/include/core/Resource.h | 0
.../include/core/StreamManager.h | 0
.../include/core/ThreadedRepository.h | 0
.../include/core/TypedValues.h | 0
.../include/core/VariableRegistry.h | 0
.../include/core/controller/ControllerService.h | 0
{utils => core-framework}/include/core/expect.h | 0
.../include/core/extension/Extension.h | 0
.../include/core/logging/Logger.h | 0
.../include/core/logging/LoggerFactory.h | 0
.../include/core/state/MetricsPublisher.h | 0
.../include/core/state/PublishedMetricProvider.h | 0
.../include/core/state/Value.h | 0
.../include/core/state/nodes/ResponseNode.h | 0
.../include/http/BaseHTTPClient.h | 0
.../include/http/HTTPCallback.h | 0
.../include/http/HTTPClient.h | 0
.../include/http/HTTPStream.h | 0
.../include/io/ArchiveStream.h | 0
{utils => core-framework}/include/io/AsioStream.h | 0
{utils => core-framework}/include/io/BaseStream.h | 0
.../include/io/BufferStream.h | 0
{utils => core-framework}/include/io/CRCStream.h | 0
{utils => core-framework}/include/io/FileStream.h | 0
{utils => core-framework}/include/io/InputStream.h | 0
.../include/io/OutputStream.h | 0
{utils => core-framework}/include/io/Stream.h | 0
{utils => core-framework}/include/io/StreamPipe.h | 0
{utils => core-framework}/include/io/StreamSlice.h | 0
{utils => core-framework}/include/io/ZlibStream.h | 0
{utils => core-framework}/include/io/validation.h | 0
.../include/utils/ArrayUtils.h | 0
.../include/utils/AttributeErrors.h | 0
.../include/utils/AutoPersistor.h | 0
.../include/utils/BackTrace.h | 0
.../include/utils/ByteArrayCallback.h | 0
.../include/utils/CallBackTimer.h | 0
.../include/utils/ClassUtils.h | 0
.../include/utils/CollectionUtils.h | 0
.../include/utils/ConfigurationUtils.h | 0
{utils => core-framework}/include/utils/Cron.h | 0
{utils => core-framework}/include/utils/Deleters.h | 0
{utils => core-framework}/include/utils/Enum.h | 0
.../include/utils/Environment.h | 0
{utils => core-framework}/include/utils/Error.h | 0
{utils => core-framework}/include/utils/Export.h | 0
.../include/utils/FailurePolicy.h | 0
.../include/utils/FifoExecutor.h | 0
.../include/utils/FileMutex.h | 0
.../include/utils/GeneralUtils.h | 0
.../include/utils/HTTPUtils.h | 0
{utils => core-framework}/include/utils/Hash.h | 0
{utils => core-framework}/include/utils/Id.h | 0
.../include/utils/IntervalSwitch.h | 0
.../include/utils/JsonCallback.h | 0
.../utils/LineByLineInputOutputStreamCallback.h | 0
{utils => core-framework}/include/utils/Literals.h | 0
{utils => core-framework}/include/utils/LogUtils.h | 0
{utils => core-framework}/include/utils/Macro.h | 0
{utils => core-framework}/include/utils/MapUtils.h | 0
.../include/utils/MinifiConcurrentQueue.h | 0
{utils => core-framework}/include/utils/Monitors.h | 0
.../include/utils/OptionalUtils.h | 0
{utils => core-framework}/include/utils/OsUtils.h | 0
.../include/utils/ParsingErrors.h | 0
.../include/utils/ParsingUtils.h | 0
.../include/utils/ProcessCpuUsageTracker.h | 0
.../include/utils/PropertyErrors.h | 0
.../include/utils/PropertyExceptions.h | 0
.../include/utils/RegexUtils.h | 0
{utils => core-framework}/include/utils/Searcher.h | 0
.../include/utils/SmallString.h | 0
.../include/utils/StoppableThread.h | 0
.../include/utils/StringUtils.h | 0
.../include/utils/SystemCpuUsageTracker.h | 0
.../include/utils/ThreadPool.h | 0
{utils => core-framework}/include/utils/TimeUtil.h | 0
.../include/utils/TryMoveCall.h | 0
.../include/utils/UnicodeConversion.h | 0
.../include/utils/ValueCaster.h | 0
.../include/utils/ValueParser.h | 0
.../include/utils/crypto/EncryptionManager.h | 0
.../include/utils/crypto/EncryptionProvider.h | 0
.../include/utils/crypto/EncryptionUtils.h | 0
.../include/utils/crypto/ciphers/Aes256Ecb.h | 0
.../include/utils/crypto/ciphers/XSalsa20.h | 0
.../property_encryption/PropertyEncryptionUtils.h | 0
.../utils/detail/MonadicOperationWrappers.h | 0
{utils => core-framework}/include/utils/expected.h | 0
.../include/utils/file/FilePattern.h | 0
.../include/utils/file/FileUtils.h | 0
.../include/utils/file/PathUtils.h | 0
{utils => core-framework}/include/utils/gsl.h | 0
.../include/utils/meta/detected.h | 0
.../include/utils/meta/type_list.h | 0
.../include/utils/net/AsioCoro.h | 0
.../include/utils/net/AsioSocketUtils.h | 0
{utils => core-framework}/include/utils/net/DNS.h | 0
.../include/utils/net/NetworkInterfaceInfo.h | 0
.../include/utils/net/Socket.h | 0
.../include/utils/requirements/Container.h | 0
.../utils/requirements/EqualityComparable.h | 0
.../utils/requirements/LegacyForwardIterator.h | 0
.../utils/requirements/LegacyInputIterator.h | 0
.../include/utils/requirements/LegacyIterator.h | 0
.../include/utils/requirements/Swappable.h | 0
{utils => core-framework}/include/utils/span.h | 0
.../src/core/BufferedContentSession.cpp | 0
.../src/core/ConfigurableComponentImpl.cpp | 0
{utils => core-framework}/src/core/Connectable.cpp | 0
.../src/core/ContentRepository.cpp | 0
.../src/core/ContentSession.cpp | 0
{utils => core-framework}/src/core/Core.cpp | 0
.../src/core/ProcessContextBuilder.cpp | 0
.../src/core/ProcessSession.cpp | 0
.../src/core/ProcessorImpl.cpp | 0
.../src/core/ProcessorMetrics.cpp | 0
{utils => core-framework}/src/core/Property.cpp | 0
{utils => core-framework}/src/core/Repository.cpp | 0
{utils => core-framework}/src/core/TypedValues.cpp | 0
.../src/core/extension/Extension.cpp | 0
.../src/http/BaseHTTPClient.cpp | 0
{utils => core-framework}/src/http/HTTPClient.cpp | 0
{utils => core-framework}/src/http/HTTPStream.cpp | 0
{utils => core-framework}/src/io/BufferStream.cpp | 0
{utils => core-framework}/src/io/CPPLINT.cfg | 0
{utils => core-framework}/src/io/CRCStream.cpp | 0
{utils => core-framework}/src/io/FileStream.cpp | 0
{utils => core-framework}/src/io/InputStream.cpp | 0
{utils => core-framework}/src/io/OutputStream.cpp | 0
{utils => core-framework}/src/io/StreamSlice.cpp | 0
{utils => core-framework}/src/io/ZlibStream.cpp | 0
.../src/utils/AutoPersistor.cpp | 0
{utils => core-framework}/src/utils/BackTrace.cpp | 0
.../src/utils/ByteArrayCallback.cpp | 0
{utils => core-framework}/src/utils/ClassUtils.cpp | 0
.../src/utils/ConfigurationUtils.cpp | 0
{utils => core-framework}/src/utils/Cron.cpp | 0
.../src/utils/Environment.cpp | 0
{utils => core-framework}/src/utils/Error.cpp | 0
.../src/utils/FifoExecutor.cpp | 0
{utils => core-framework}/src/utils/FileMutex.cpp | 0
{utils => core-framework}/src/utils/HTTPUtils.cpp | 0
{utils => core-framework}/src/utils/Id.cpp | 0
.../utils/LineByLineInputOutputStreamCallback.cpp | 0
{utils => core-framework}/src/utils/OsUtils.cpp | 0
.../src/utils/ParsingUtils.cpp | 0
.../src/utils/ProcessCpuUsageTracker.cpp | 0
{utils => core-framework}/src/utils/RegexUtils.cpp | 0
.../src/utils/StoppableThread.cpp | 0
.../src/utils/StringUtils.cpp | 0
.../src/utils/SystemCpuUsageTracker.cpp | 0
{utils => core-framework}/src/utils/ThreadPool.cpp | 0
{utils => core-framework}/src/utils/TimeUtil.cpp | 0
.../src/utils/crypto/EncryptionManager.cpp | 0
.../src/utils/crypto/EncryptionProvider.cpp | 0
.../src/utils/crypto/EncryptionUtils.cpp | 0
.../src/utils/crypto/ciphers/Aes256Ecb.cpp | 0
.../PropertyEncryptionUtils.cpp | 0
.../src/utils/file/FilePattern.cpp | 0
.../src/utils/file/FileUtils.cpp | 0
.../src/utils/file/PathUtils.cpp | 0
.../src/utils/net/AsioSocketUtils.cpp | 0
{utils => core-framework}/src/utils/net/DNS.cpp | 0
.../src/utils/net/NetworkInterfaceInfo.cpp | 0
{utils => core-framework}/src/utils/net/Socket.cpp | 0
.../CMakeLists.txt | 16 ++++++++--------
.../include/FlowFileRecord.h | 0
.../include/ResourceClaim.h | 0
.../include/controllers/AttributeProviderService.h | 0
.../include/controllers/RecordSetReader.h | 0
.../include/controllers/RecordSetWriter.h | 0
.../include/controllers/SSLContextService.h | 0
.../controllers/SSLContextServiceInterface.h | 0
.../controllers/keyvalue/KeyValueStateManager.h | 0
.../controllers/keyvalue/KeyValueStateStorage.h | 0
.../include/core/AbstractProcessor.h | 0
.../include/core/FlowFile.h | 0
.../include/core/FlowFileStore.h | 0
.../include/core/ProcessSession.h | 0
.../include/core/StateManager.h | 0
.../include/core/StateStorage.h | 0
.../include/properties/Configure.h | 0
.../include/properties/Properties.h | 0
.../include/serialization/FlowFileSerializer.h | 0
.../include/serialization/FlowFileV3Serializer.h | 0
.../include/serialization/PayloadSerializer.h | 0
.../include/utils/ListingStateManager.h | 0
.../include/utils/OpenTelemetryLogDataModelUtils.h | 0
.../include/utils/ProcessorConfigUtils.h | 0
.../include/utils/ResourceQueue.h | 0
.../include/utils/file/FileManager.h | 0
.../include/utils/file/FileReaderCallback.h | 0
.../include/utils/file/FileWriterCallback.h | 0
.../include/utils/file/ListedFile.h | 0
.../include/utils/net/ConnectionHandler.h | 0
.../include/utils/net/ConnectionHandlerBase.h | 0
.../include/utils/net/IpProtocol.h | 0
.../include/utils/net/Message.h | 0
.../include/utils/net/Server.h | 0
.../include/utils/net/Ssl.h | 0
.../include/utils/net/TcpServer.h | 0
.../include/utils/net/UdpServer.h | 0
.../controllers/keyvalue/KeyValueStateManager.cpp | 0
.../controllers/keyvalue/KeyValueStateStorage.cpp | 0
.../src/core/ProcessContext.cpp | 0
.../src/serialization/FlowFileV3Serializer.cpp | 0
.../src/serialization/PayloadSerializer.cpp | 0
.../src/utils/CallBackTimer.cpp | 0
.../src/utils/ListingStateManager.cpp | 0
.../src/utils/file/FileReaderCallback.cpp | 0
.../src/utils/file/FileWriterCallback.cpp | 0
.../src/utils/net/Ssl.cpp | 0
.../src/utils/net/TcpServer.cpp | 0
.../src/utils/net/UdpServer.cpp | 0
.../include/controllers/SSLContextService.h | 0
extensions/rocksdb-repos/CMakeLists.txt | 2 +-
libminifi/CMakeLists.txt | 4 ++--
libminifi/test/libtest/unit/CMakeLists.txt | 2 +-
utils/CMakeLists.txt | 22 ----------------------
249 files changed, 36 insertions(+), 36 deletions(-)
diff --git a/CMakeLists.txt b/CMakeLists.txt
index f9f94363e..7748ac0e5 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -339,8 +339,8 @@ SET(TEST_DIR ${CMAKE_SOURCE_DIR}/libminifi/test)
include(Extensions)
add_subdirectory(minifi-api)
-add_subdirectory(utils)
-add_subdirectory(extension-utils)
+add_subdirectory(core-framework)
+add_subdirectory(extension-framework)
add_subdirectory(libminifi)
if (ENABLE_ALL OR ENABLE_AZURE)
diff --git a/core-framework/CMakeLists.txt b/core-framework/CMakeLists.txt
new file mode 100644
index 000000000..cf3203f24
--- /dev/null
+++ b/core-framework/CMakeLists.txt
@@ -0,0 +1,22 @@
+file(GLOB SOURCES
+ src/*.cpp
+ src/core/*.cpp
+ src/core/extension/*.cpp
+ src/io/*.cpp
+ src/http/*.cpp
+ src/utils/*.cpp
+ src/utils/crypto/*.cpp
+ src/utils/crypto/ciphers/*.cpp
+ src/utils/crypto/property_encryption/*.cpp
+ src/utils/net/*.cpp
+ src/utils/file/*.cpp)
+
+add_minifi_library(minifi-core-framework STATIC ${SOURCES})
+target_include_directories(minifi-core-framework PUBLIC include)
+target_link_libraries(minifi-core-framework PUBLIC minifi-api ZLIB::ZLIB concurrentqueue RapidJSON spdlog Threads::Threads gsl-lite libsodium range-v3 expected-lite date::date date::tz asio magic_enum OpenSSL::Crypto OpenSSL::SSL CURL::libcurl RapidJSON)
+if(NOT WIN32)
+ target_link_libraries(minifi-core-framework PUBLIC OSSP::libuuid++)
+endif()
+if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
+ target_link_libraries(minifi-core-framework PUBLIC stdc++fs)
+endif()
diff --git a/utils/include/Exception.h b/core-framework/include/Exception.h
similarity index 100%
rename from utils/include/Exception.h
rename to core-framework/include/Exception.h
diff --git a/utils/include/SwapManager.h b/core-framework/include/SwapManager.h
similarity index 100%
rename from utils/include/SwapManager.h
rename to core-framework/include/SwapManager.h
diff --git a/utils/include/agent/agent_docs.h b/core-framework/include/agent/agent_docs.h
similarity index 100%
rename from utils/include/agent/agent_docs.h
rename to core-framework/include/agent/agent_docs.h
diff --git a/utils/include/agent/agent_version.h b/core-framework/include/agent/agent_version.h
similarity index 100%
rename from utils/include/agent/agent_version.h
rename to core-framework/include/agent/agent_version.h
diff --git a/utils/include/core/Annotation.h b/core-framework/include/core/Annotation.h
similarity index 100%
rename from utils/include/core/Annotation.h
rename to core-framework/include/core/Annotation.h
diff --git a/utils/include/core/BufferedContentSession.h b/core-framework/include/core/BufferedContentSession.h
similarity index 100%
rename from utils/include/core/BufferedContentSession.h
rename to core-framework/include/core/BufferedContentSession.h
diff --git a/utils/include/core/ClassLoader.h b/core-framework/include/core/ClassLoader.h
similarity index 100%
rename from utils/include/core/ClassLoader.h
rename to core-framework/include/core/ClassLoader.h
diff --git a/utils/include/core/ClassName.h b/core-framework/include/core/ClassName.h
similarity index 100%
rename from utils/include/core/ClassName.h
rename to core-framework/include/core/ClassName.h
diff --git a/utils/include/core/ConfigurableComponentImpl.h b/core-framework/include/core/ConfigurableComponentImpl.h
similarity index 100%
rename from utils/include/core/ConfigurableComponentImpl.h
rename to core-framework/include/core/ConfigurableComponentImpl.h
diff --git a/utils/include/core/Connectable.h b/core-framework/include/core/Connectable.h
similarity index 100%
rename from utils/include/core/Connectable.h
rename to core-framework/include/core/Connectable.h
diff --git a/utils/include/core/ContentRepository.h b/core-framework/include/core/ContentRepository.h
similarity index 100%
rename from utils/include/core/ContentRepository.h
rename to core-framework/include/core/ContentRepository.h
diff --git a/utils/include/core/ContentSession.h b/core-framework/include/core/ContentSession.h
similarity index 100%
rename from utils/include/core/ContentSession.h
rename to core-framework/include/core/ContentSession.h
diff --git a/utils/include/core/Core.h b/core-framework/include/core/Core.h
similarity index 100%
rename from utils/include/core/Core.h
rename to core-framework/include/core/Core.h
diff --git a/utils/include/core/ObjectFactory.h b/core-framework/include/core/ObjectFactory.h
similarity index 100%
rename from utils/include/core/ObjectFactory.h
rename to core-framework/include/core/ObjectFactory.h
diff --git a/utils/include/core/OutputAttributeDefinition.h b/core-framework/include/core/OutputAttributeDefinition.h
similarity index 100%
rename from utils/include/core/OutputAttributeDefinition.h
rename to core-framework/include/core/OutputAttributeDefinition.h
diff --git a/utils/include/core/ProcessContext.h b/core-framework/include/core/ProcessContext.h
similarity index 100%
rename from utils/include/core/ProcessContext.h
rename to core-framework/include/core/ProcessContext.h
diff --git a/utils/include/core/ProcessContextBuilder.h b/core-framework/include/core/ProcessContextBuilder.h
similarity index 100%
rename from utils/include/core/ProcessContextBuilder.h
rename to core-framework/include/core/ProcessContextBuilder.h
diff --git a/utils/include/core/ProcessorFactory.h b/core-framework/include/core/ProcessorFactory.h
similarity index 100%
rename from utils/include/core/ProcessorFactory.h
rename to core-framework/include/core/ProcessorFactory.h
diff --git a/utils/include/core/ProcessorImpl.h b/core-framework/include/core/ProcessorImpl.h
similarity index 100%
rename from utils/include/core/ProcessorImpl.h
rename to core-framework/include/core/ProcessorImpl.h
diff --git a/utils/include/core/ProcessorMetrics.h b/core-framework/include/core/ProcessorMetrics.h
similarity index 100%
rename from utils/include/core/ProcessorMetrics.h
rename to core-framework/include/core/ProcessorMetrics.h
diff --git a/utils/include/core/Property.h b/core-framework/include/core/Property.h
similarity index 100%
rename from utils/include/core/Property.h
rename to core-framework/include/core/Property.h
diff --git a/utils/include/core/PropertyDefinition.h b/core-framework/include/core/PropertyDefinition.h
similarity index 100%
rename from utils/include/core/PropertyDefinition.h
rename to core-framework/include/core/PropertyDefinition.h
diff --git a/utils/include/core/PropertyDefinitionBuilder.h b/core-framework/include/core/PropertyDefinitionBuilder.h
similarity index 100%
rename from utils/include/core/PropertyDefinitionBuilder.h
rename to core-framework/include/core/PropertyDefinitionBuilder.h
diff --git a/utils/include/core/Relationship.h b/core-framework/include/core/Relationship.h
similarity index 100%
rename from utils/include/core/Relationship.h
rename to core-framework/include/core/Relationship.h
diff --git a/utils/include/core/RelationshipDefinition.h b/core-framework/include/core/RelationshipDefinition.h
similarity index 100%
rename from utils/include/core/RelationshipDefinition.h
rename to core-framework/include/core/RelationshipDefinition.h
diff --git a/utils/include/core/Repository.h b/core-framework/include/core/Repository.h
similarity index 100%
rename from utils/include/core/Repository.h
rename to core-framework/include/core/Repository.h
diff --git a/utils/include/core/RepositoryMetricsSource.h b/core-framework/include/core/RepositoryMetricsSource.h
similarity index 100%
rename from utils/include/core/RepositoryMetricsSource.h
rename to core-framework/include/core/RepositoryMetricsSource.h
diff --git a/utils/include/core/Resource.h b/core-framework/include/core/Resource.h
similarity index 100%
rename from utils/include/core/Resource.h
rename to core-framework/include/core/Resource.h
diff --git a/utils/include/core/StreamManager.h b/core-framework/include/core/StreamManager.h
similarity index 100%
rename from utils/include/core/StreamManager.h
rename to core-framework/include/core/StreamManager.h
diff --git a/utils/include/core/ThreadedRepository.h b/core-framework/include/core/ThreadedRepository.h
similarity index 100%
rename from utils/include/core/ThreadedRepository.h
rename to core-framework/include/core/ThreadedRepository.h
diff --git a/utils/include/core/TypedValues.h b/core-framework/include/core/TypedValues.h
similarity index 100%
rename from utils/include/core/TypedValues.h
rename to core-framework/include/core/TypedValues.h
diff --git a/utils/include/core/VariableRegistry.h b/core-framework/include/core/VariableRegistry.h
similarity index 100%
rename from utils/include/core/VariableRegistry.h
rename to core-framework/include/core/VariableRegistry.h
diff --git a/utils/include/core/controller/ControllerService.h b/core-framework/include/core/controller/ControllerService.h
similarity index 100%
rename from utils/include/core/controller/ControllerService.h
rename to core-framework/include/core/controller/ControllerService.h
diff --git a/utils/include/core/expect.h b/core-framework/include/core/expect.h
similarity index 100%
rename from utils/include/core/expect.h
rename to core-framework/include/core/expect.h
diff --git a/utils/include/core/extension/Extension.h b/core-framework/include/core/extension/Extension.h
similarity index 100%
rename from utils/include/core/extension/Extension.h
rename to core-framework/include/core/extension/Extension.h
diff --git a/utils/include/core/logging/Logger.h b/core-framework/include/core/logging/Logger.h
similarity index 100%
rename from utils/include/core/logging/Logger.h
rename to core-framework/include/core/logging/Logger.h
diff --git a/utils/include/core/logging/LoggerFactory.h b/core-framework/include/core/logging/LoggerFactory.h
similarity index 100%
rename from utils/include/core/logging/LoggerFactory.h
rename to core-framework/include/core/logging/LoggerFactory.h
diff --git a/utils/include/core/state/MetricsPublisher.h b/core-framework/include/core/state/MetricsPublisher.h
similarity index 100%
rename from utils/include/core/state/MetricsPublisher.h
rename to core-framework/include/core/state/MetricsPublisher.h
diff --git a/utils/include/core/state/PublishedMetricProvider.h b/core-framework/include/core/state/PublishedMetricProvider.h
similarity index 100%
rename from utils/include/core/state/PublishedMetricProvider.h
rename to core-framework/include/core/state/PublishedMetricProvider.h
diff --git a/utils/include/core/state/Value.h b/core-framework/include/core/state/Value.h
similarity index 100%
rename from utils/include/core/state/Value.h
rename to core-framework/include/core/state/Value.h
diff --git a/utils/include/core/state/nodes/ResponseNode.h b/core-framework/include/core/state/nodes/ResponseNode.h
similarity index 100%
rename from utils/include/core/state/nodes/ResponseNode.h
rename to core-framework/include/core/state/nodes/ResponseNode.h
diff --git a/utils/include/http/BaseHTTPClient.h b/core-framework/include/http/BaseHTTPClient.h
similarity index 100%
rename from utils/include/http/BaseHTTPClient.h
rename to core-framework/include/http/BaseHTTPClient.h
diff --git a/utils/include/http/HTTPCallback.h b/core-framework/include/http/HTTPCallback.h
similarity index 100%
rename from utils/include/http/HTTPCallback.h
rename to core-framework/include/http/HTTPCallback.h
diff --git a/utils/include/http/HTTPClient.h b/core-framework/include/http/HTTPClient.h
similarity index 100%
rename from utils/include/http/HTTPClient.h
rename to core-framework/include/http/HTTPClient.h
diff --git a/utils/include/http/HTTPStream.h b/core-framework/include/http/HTTPStream.h
similarity index 100%
rename from utils/include/http/HTTPStream.h
rename to core-framework/include/http/HTTPStream.h
diff --git a/utils/include/io/ArchiveStream.h b/core-framework/include/io/ArchiveStream.h
similarity index 100%
rename from utils/include/io/ArchiveStream.h
rename to core-framework/include/io/ArchiveStream.h
diff --git a/utils/include/io/AsioStream.h b/core-framework/include/io/AsioStream.h
similarity index 100%
rename from utils/include/io/AsioStream.h
rename to core-framework/include/io/AsioStream.h
diff --git a/utils/include/io/BaseStream.h b/core-framework/include/io/BaseStream.h
similarity index 100%
rename from utils/include/io/BaseStream.h
rename to core-framework/include/io/BaseStream.h
diff --git a/utils/include/io/BufferStream.h b/core-framework/include/io/BufferStream.h
similarity index 100%
rename from utils/include/io/BufferStream.h
rename to core-framework/include/io/BufferStream.h
diff --git a/utils/include/io/CRCStream.h b/core-framework/include/io/CRCStream.h
similarity index 100%
rename from utils/include/io/CRCStream.h
rename to core-framework/include/io/CRCStream.h
diff --git a/utils/include/io/FileStream.h b/core-framework/include/io/FileStream.h
similarity index 100%
rename from utils/include/io/FileStream.h
rename to core-framework/include/io/FileStream.h
diff --git a/utils/include/io/InputStream.h b/core-framework/include/io/InputStream.h
similarity index 100%
rename from utils/include/io/InputStream.h
rename to core-framework/include/io/InputStream.h
diff --git a/utils/include/io/OutputStream.h b/core-framework/include/io/OutputStream.h
similarity index 100%
rename from utils/include/io/OutputStream.h
rename to core-framework/include/io/OutputStream.h
diff --git a/utils/include/io/Stream.h b/core-framework/include/io/Stream.h
similarity index 100%
rename from utils/include/io/Stream.h
rename to core-framework/include/io/Stream.h
diff --git a/utils/include/io/StreamPipe.h b/core-framework/include/io/StreamPipe.h
similarity index 100%
rename from utils/include/io/StreamPipe.h
rename to core-framework/include/io/StreamPipe.h
diff --git a/utils/include/io/StreamSlice.h b/core-framework/include/io/StreamSlice.h
similarity index 100%
rename from utils/include/io/StreamSlice.h
rename to core-framework/include/io/StreamSlice.h
diff --git a/utils/include/io/ZlibStream.h b/core-framework/include/io/ZlibStream.h
similarity index 100%
rename from utils/include/io/ZlibStream.h
rename to core-framework/include/io/ZlibStream.h
diff --git a/utils/include/io/validation.h b/core-framework/include/io/validation.h
similarity index 100%
rename from utils/include/io/validation.h
rename to core-framework/include/io/validation.h
diff --git a/utils/include/utils/ArrayUtils.h b/core-framework/include/utils/ArrayUtils.h
similarity index 100%
rename from utils/include/utils/ArrayUtils.h
rename to core-framework/include/utils/ArrayUtils.h
diff --git a/utils/include/utils/AttributeErrors.h b/core-framework/include/utils/AttributeErrors.h
similarity index 100%
rename from utils/include/utils/AttributeErrors.h
rename to core-framework/include/utils/AttributeErrors.h
diff --git a/utils/include/utils/AutoPersistor.h b/core-framework/include/utils/AutoPersistor.h
similarity index 100%
rename from utils/include/utils/AutoPersistor.h
rename to core-framework/include/utils/AutoPersistor.h
diff --git a/utils/include/utils/BackTrace.h b/core-framework/include/utils/BackTrace.h
similarity index 100%
rename from utils/include/utils/BackTrace.h
rename to core-framework/include/utils/BackTrace.h
diff --git a/utils/include/utils/ByteArrayCallback.h b/core-framework/include/utils/ByteArrayCallback.h
similarity index 100%
rename from utils/include/utils/ByteArrayCallback.h
rename to core-framework/include/utils/ByteArrayCallback.h
diff --git a/utils/include/utils/CallBackTimer.h b/core-framework/include/utils/CallBackTimer.h
similarity index 100%
rename from utils/include/utils/CallBackTimer.h
rename to core-framework/include/utils/CallBackTimer.h
diff --git a/utils/include/utils/ClassUtils.h b/core-framework/include/utils/ClassUtils.h
similarity index 100%
rename from utils/include/utils/ClassUtils.h
rename to core-framework/include/utils/ClassUtils.h
diff --git a/utils/include/utils/CollectionUtils.h b/core-framework/include/utils/CollectionUtils.h
similarity index 100%
rename from utils/include/utils/CollectionUtils.h
rename to core-framework/include/utils/CollectionUtils.h
diff --git a/utils/include/utils/ConfigurationUtils.h b/core-framework/include/utils/ConfigurationUtils.h
similarity index 100%
rename from utils/include/utils/ConfigurationUtils.h
rename to core-framework/include/utils/ConfigurationUtils.h
diff --git a/utils/include/utils/Cron.h b/core-framework/include/utils/Cron.h
similarity index 100%
rename from utils/include/utils/Cron.h
rename to core-framework/include/utils/Cron.h
diff --git a/utils/include/utils/Deleters.h b/core-framework/include/utils/Deleters.h
similarity index 100%
rename from utils/include/utils/Deleters.h
rename to core-framework/include/utils/Deleters.h
diff --git a/utils/include/utils/Enum.h b/core-framework/include/utils/Enum.h
similarity index 100%
rename from utils/include/utils/Enum.h
rename to core-framework/include/utils/Enum.h
diff --git a/utils/include/utils/Environment.h b/core-framework/include/utils/Environment.h
similarity index 100%
rename from utils/include/utils/Environment.h
rename to core-framework/include/utils/Environment.h
diff --git a/utils/include/utils/Error.h b/core-framework/include/utils/Error.h
similarity index 100%
rename from utils/include/utils/Error.h
rename to core-framework/include/utils/Error.h
diff --git a/utils/include/utils/Export.h b/core-framework/include/utils/Export.h
similarity index 100%
rename from utils/include/utils/Export.h
rename to core-framework/include/utils/Export.h
diff --git a/utils/include/utils/FailurePolicy.h b/core-framework/include/utils/FailurePolicy.h
similarity index 100%
rename from utils/include/utils/FailurePolicy.h
rename to core-framework/include/utils/FailurePolicy.h
diff --git a/utils/include/utils/FifoExecutor.h b/core-framework/include/utils/FifoExecutor.h
similarity index 100%
rename from utils/include/utils/FifoExecutor.h
rename to core-framework/include/utils/FifoExecutor.h
diff --git a/utils/include/utils/FileMutex.h b/core-framework/include/utils/FileMutex.h
similarity index 100%
rename from utils/include/utils/FileMutex.h
rename to core-framework/include/utils/FileMutex.h
diff --git a/utils/include/utils/GeneralUtils.h b/core-framework/include/utils/GeneralUtils.h
similarity index 100%
rename from utils/include/utils/GeneralUtils.h
rename to core-framework/include/utils/GeneralUtils.h
diff --git a/utils/include/utils/HTTPUtils.h b/core-framework/include/utils/HTTPUtils.h
similarity index 100%
rename from utils/include/utils/HTTPUtils.h
rename to core-framework/include/utils/HTTPUtils.h
diff --git a/utils/include/utils/Hash.h b/core-framework/include/utils/Hash.h
similarity index 100%
rename from utils/include/utils/Hash.h
rename to core-framework/include/utils/Hash.h
diff --git a/utils/include/utils/Id.h b/core-framework/include/utils/Id.h
similarity index 100%
rename from utils/include/utils/Id.h
rename to core-framework/include/utils/Id.h
diff --git a/utils/include/utils/IntervalSwitch.h b/core-framework/include/utils/IntervalSwitch.h
similarity index 100%
rename from utils/include/utils/IntervalSwitch.h
rename to core-framework/include/utils/IntervalSwitch.h
diff --git a/utils/include/utils/JsonCallback.h b/core-framework/include/utils/JsonCallback.h
similarity index 100%
rename from utils/include/utils/JsonCallback.h
rename to core-framework/include/utils/JsonCallback.h
diff --git a/utils/include/utils/LineByLineInputOutputStreamCallback.h b/core-framework/include/utils/LineByLineInputOutputStreamCallback.h
similarity index 100%
rename from utils/include/utils/LineByLineInputOutputStreamCallback.h
rename to core-framework/include/utils/LineByLineInputOutputStreamCallback.h
diff --git a/utils/include/utils/Literals.h b/core-framework/include/utils/Literals.h
similarity index 100%
rename from utils/include/utils/Literals.h
rename to core-framework/include/utils/Literals.h
diff --git a/utils/include/utils/LogUtils.h b/core-framework/include/utils/LogUtils.h
similarity index 100%
rename from utils/include/utils/LogUtils.h
rename to core-framework/include/utils/LogUtils.h
diff --git a/utils/include/utils/Macro.h b/core-framework/include/utils/Macro.h
similarity index 100%
rename from utils/include/utils/Macro.h
rename to core-framework/include/utils/Macro.h
diff --git a/utils/include/utils/MapUtils.h b/core-framework/include/utils/MapUtils.h
similarity index 100%
rename from utils/include/utils/MapUtils.h
rename to core-framework/include/utils/MapUtils.h
diff --git a/utils/include/utils/MinifiConcurrentQueue.h b/core-framework/include/utils/MinifiConcurrentQueue.h
similarity index 100%
rename from utils/include/utils/MinifiConcurrentQueue.h
rename to core-framework/include/utils/MinifiConcurrentQueue.h
diff --git a/utils/include/utils/Monitors.h b/core-framework/include/utils/Monitors.h
similarity index 100%
rename from utils/include/utils/Monitors.h
rename to core-framework/include/utils/Monitors.h
diff --git a/utils/include/utils/OptionalUtils.h b/core-framework/include/utils/OptionalUtils.h
similarity index 100%
rename from utils/include/utils/OptionalUtils.h
rename to core-framework/include/utils/OptionalUtils.h
diff --git a/utils/include/utils/OsUtils.h b/core-framework/include/utils/OsUtils.h
similarity index 100%
rename from utils/include/utils/OsUtils.h
rename to core-framework/include/utils/OsUtils.h
diff --git a/utils/include/utils/ParsingErrors.h b/core-framework/include/utils/ParsingErrors.h
similarity index 100%
rename from utils/include/utils/ParsingErrors.h
rename to core-framework/include/utils/ParsingErrors.h
diff --git a/utils/include/utils/ParsingUtils.h b/core-framework/include/utils/ParsingUtils.h
similarity index 100%
rename from utils/include/utils/ParsingUtils.h
rename to core-framework/include/utils/ParsingUtils.h
diff --git a/utils/include/utils/ProcessCpuUsageTracker.h b/core-framework/include/utils/ProcessCpuUsageTracker.h
similarity index 100%
rename from utils/include/utils/ProcessCpuUsageTracker.h
rename to core-framework/include/utils/ProcessCpuUsageTracker.h
diff --git a/utils/include/utils/PropertyErrors.h b/core-framework/include/utils/PropertyErrors.h
similarity index 100%
rename from utils/include/utils/PropertyErrors.h
rename to core-framework/include/utils/PropertyErrors.h
diff --git a/utils/include/utils/PropertyExceptions.h b/core-framework/include/utils/PropertyExceptions.h
similarity index 100%
rename from utils/include/utils/PropertyExceptions.h
rename to core-framework/include/utils/PropertyExceptions.h
diff --git a/utils/include/utils/RegexUtils.h b/core-framework/include/utils/RegexUtils.h
similarity index 100%
rename from utils/include/utils/RegexUtils.h
rename to core-framework/include/utils/RegexUtils.h
diff --git a/utils/include/utils/Searcher.h b/core-framework/include/utils/Searcher.h
similarity index 100%
rename from utils/include/utils/Searcher.h
rename to core-framework/include/utils/Searcher.h
diff --git a/utils/include/utils/SmallString.h b/core-framework/include/utils/SmallString.h
similarity index 100%
rename from utils/include/utils/SmallString.h
rename to core-framework/include/utils/SmallString.h
diff --git a/utils/include/utils/StoppableThread.h b/core-framework/include/utils/StoppableThread.h
similarity index 100%
rename from utils/include/utils/StoppableThread.h
rename to core-framework/include/utils/StoppableThread.h
diff --git a/utils/include/utils/StringUtils.h b/core-framework/include/utils/StringUtils.h
similarity index 100%
rename from utils/include/utils/StringUtils.h
rename to core-framework/include/utils/StringUtils.h
diff --git a/utils/include/utils/SystemCpuUsageTracker.h b/core-framework/include/utils/SystemCpuUsageTracker.h
similarity index 100%
rename from utils/include/utils/SystemCpuUsageTracker.h
rename to core-framework/include/utils/SystemCpuUsageTracker.h
diff --git a/utils/include/utils/ThreadPool.h b/core-framework/include/utils/ThreadPool.h
similarity index 100%
rename from utils/include/utils/ThreadPool.h
rename to core-framework/include/utils/ThreadPool.h
diff --git a/utils/include/utils/TimeUtil.h b/core-framework/include/utils/TimeUtil.h
similarity index 100%
rename from utils/include/utils/TimeUtil.h
rename to core-framework/include/utils/TimeUtil.h
diff --git a/utils/include/utils/TryMoveCall.h b/core-framework/include/utils/TryMoveCall.h
similarity index 100%
rename from utils/include/utils/TryMoveCall.h
rename to core-framework/include/utils/TryMoveCall.h
diff --git a/utils/include/utils/UnicodeConversion.h b/core-framework/include/utils/UnicodeConversion.h
similarity index 100%
rename from utils/include/utils/UnicodeConversion.h
rename to core-framework/include/utils/UnicodeConversion.h
diff --git a/utils/include/utils/ValueCaster.h b/core-framework/include/utils/ValueCaster.h
similarity index 100%
rename from utils/include/utils/ValueCaster.h
rename to core-framework/include/utils/ValueCaster.h
diff --git a/utils/include/utils/ValueParser.h b/core-framework/include/utils/ValueParser.h
similarity index 100%
rename from utils/include/utils/ValueParser.h
rename to core-framework/include/utils/ValueParser.h
diff --git a/utils/include/utils/crypto/EncryptionManager.h b/core-framework/include/utils/crypto/EncryptionManager.h
similarity index 100%
rename from utils/include/utils/crypto/EncryptionManager.h
rename to core-framework/include/utils/crypto/EncryptionManager.h
diff --git a/utils/include/utils/crypto/EncryptionProvider.h b/core-framework/include/utils/crypto/EncryptionProvider.h
similarity index 100%
rename from utils/include/utils/crypto/EncryptionProvider.h
rename to core-framework/include/utils/crypto/EncryptionProvider.h
diff --git a/utils/include/utils/crypto/EncryptionUtils.h b/core-framework/include/utils/crypto/EncryptionUtils.h
similarity index 100%
rename from utils/include/utils/crypto/EncryptionUtils.h
rename to core-framework/include/utils/crypto/EncryptionUtils.h
diff --git a/utils/include/utils/crypto/ciphers/Aes256Ecb.h b/core-framework/include/utils/crypto/ciphers/Aes256Ecb.h
similarity index 100%
rename from utils/include/utils/crypto/ciphers/Aes256Ecb.h
rename to core-framework/include/utils/crypto/ciphers/Aes256Ecb.h
diff --git a/utils/include/utils/crypto/ciphers/XSalsa20.h b/core-framework/include/utils/crypto/ciphers/XSalsa20.h
similarity index 100%
rename from utils/include/utils/crypto/ciphers/XSalsa20.h
rename to core-framework/include/utils/crypto/ciphers/XSalsa20.h
diff --git a/utils/include/utils/crypto/property_encryption/PropertyEncryptionUtils.h b/core-framework/include/utils/crypto/property_encryption/PropertyEncryptionUtils.h
similarity index 100%
rename from utils/include/utils/crypto/property_encryption/PropertyEncryptionUtils.h
rename to core-framework/include/utils/crypto/property_encryption/PropertyEncryptionUtils.h
diff --git a/utils/include/utils/detail/MonadicOperationWrappers.h b/core-framework/include/utils/detail/MonadicOperationWrappers.h
similarity index 100%
rename from utils/include/utils/detail/MonadicOperationWrappers.h
rename to core-framework/include/utils/detail/MonadicOperationWrappers.h
diff --git a/utils/include/utils/expected.h b/core-framework/include/utils/expected.h
similarity index 100%
rename from utils/include/utils/expected.h
rename to core-framework/include/utils/expected.h
diff --git a/utils/include/utils/file/FilePattern.h b/core-framework/include/utils/file/FilePattern.h
similarity index 100%
rename from utils/include/utils/file/FilePattern.h
rename to core-framework/include/utils/file/FilePattern.h
diff --git a/utils/include/utils/file/FileUtils.h b/core-framework/include/utils/file/FileUtils.h
similarity index 100%
rename from utils/include/utils/file/FileUtils.h
rename to core-framework/include/utils/file/FileUtils.h
diff --git a/utils/include/utils/file/PathUtils.h b/core-framework/include/utils/file/PathUtils.h
similarity index 100%
rename from utils/include/utils/file/PathUtils.h
rename to core-framework/include/utils/file/PathUtils.h
diff --git a/utils/include/utils/gsl.h b/core-framework/include/utils/gsl.h
similarity index 100%
rename from utils/include/utils/gsl.h
rename to core-framework/include/utils/gsl.h
diff --git a/utils/include/utils/meta/detected.h b/core-framework/include/utils/meta/detected.h
similarity index 100%
rename from utils/include/utils/meta/detected.h
rename to core-framework/include/utils/meta/detected.h
diff --git a/utils/include/utils/meta/type_list.h b/core-framework/include/utils/meta/type_list.h
similarity index 100%
rename from utils/include/utils/meta/type_list.h
rename to core-framework/include/utils/meta/type_list.h
diff --git a/utils/include/utils/net/AsioCoro.h b/core-framework/include/utils/net/AsioCoro.h
similarity index 100%
rename from utils/include/utils/net/AsioCoro.h
rename to core-framework/include/utils/net/AsioCoro.h
diff --git a/utils/include/utils/net/AsioSocketUtils.h b/core-framework/include/utils/net/AsioSocketUtils.h
similarity index 100%
rename from utils/include/utils/net/AsioSocketUtils.h
rename to core-framework/include/utils/net/AsioSocketUtils.h
diff --git a/utils/include/utils/net/DNS.h b/core-framework/include/utils/net/DNS.h
similarity index 100%
rename from utils/include/utils/net/DNS.h
rename to core-framework/include/utils/net/DNS.h
diff --git a/utils/include/utils/net/NetworkInterfaceInfo.h b/core-framework/include/utils/net/NetworkInterfaceInfo.h
similarity index 100%
rename from utils/include/utils/net/NetworkInterfaceInfo.h
rename to core-framework/include/utils/net/NetworkInterfaceInfo.h
diff --git a/utils/include/utils/net/Socket.h b/core-framework/include/utils/net/Socket.h
similarity index 100%
rename from utils/include/utils/net/Socket.h
rename to core-framework/include/utils/net/Socket.h
diff --git a/utils/include/utils/requirements/Container.h b/core-framework/include/utils/requirements/Container.h
similarity index 100%
rename from utils/include/utils/requirements/Container.h
rename to core-framework/include/utils/requirements/Container.h
diff --git a/utils/include/utils/requirements/EqualityComparable.h b/core-framework/include/utils/requirements/EqualityComparable.h
similarity index 100%
rename from utils/include/utils/requirements/EqualityComparable.h
rename to core-framework/include/utils/requirements/EqualityComparable.h
diff --git a/utils/include/utils/requirements/LegacyForwardIterator.h b/core-framework/include/utils/requirements/LegacyForwardIterator.h
similarity index 100%
rename from utils/include/utils/requirements/LegacyForwardIterator.h
rename to core-framework/include/utils/requirements/LegacyForwardIterator.h
diff --git a/utils/include/utils/requirements/LegacyInputIterator.h b/core-framework/include/utils/requirements/LegacyInputIterator.h
similarity index 100%
rename from utils/include/utils/requirements/LegacyInputIterator.h
rename to core-framework/include/utils/requirements/LegacyInputIterator.h
diff --git a/utils/include/utils/requirements/LegacyIterator.h b/core-framework/include/utils/requirements/LegacyIterator.h
similarity index 100%
rename from utils/include/utils/requirements/LegacyIterator.h
rename to core-framework/include/utils/requirements/LegacyIterator.h
diff --git a/utils/include/utils/requirements/Swappable.h b/core-framework/include/utils/requirements/Swappable.h
similarity index 100%
rename from utils/include/utils/requirements/Swappable.h
rename to core-framework/include/utils/requirements/Swappable.h
diff --git a/utils/include/utils/span.h b/core-framework/include/utils/span.h
similarity index 100%
rename from utils/include/utils/span.h
rename to core-framework/include/utils/span.h
diff --git a/utils/src/core/BufferedContentSession.cpp b/core-framework/src/core/BufferedContentSession.cpp
similarity index 100%
rename from utils/src/core/BufferedContentSession.cpp
rename to core-framework/src/core/BufferedContentSession.cpp
diff --git a/utils/src/core/ConfigurableComponentImpl.cpp b/core-framework/src/core/ConfigurableComponentImpl.cpp
similarity index 100%
rename from utils/src/core/ConfigurableComponentImpl.cpp
rename to core-framework/src/core/ConfigurableComponentImpl.cpp
diff --git a/utils/src/core/Connectable.cpp b/core-framework/src/core/Connectable.cpp
similarity index 100%
rename from utils/src/core/Connectable.cpp
rename to core-framework/src/core/Connectable.cpp
diff --git a/utils/src/core/ContentRepository.cpp b/core-framework/src/core/ContentRepository.cpp
similarity index 100%
rename from utils/src/core/ContentRepository.cpp
rename to core-framework/src/core/ContentRepository.cpp
diff --git a/utils/src/core/ContentSession.cpp b/core-framework/src/core/ContentSession.cpp
similarity index 100%
rename from utils/src/core/ContentSession.cpp
rename to core-framework/src/core/ContentSession.cpp
diff --git a/utils/src/core/Core.cpp b/core-framework/src/core/Core.cpp
similarity index 100%
rename from utils/src/core/Core.cpp
rename to core-framework/src/core/Core.cpp
diff --git a/utils/src/core/ProcessContextBuilder.cpp b/core-framework/src/core/ProcessContextBuilder.cpp
similarity index 100%
rename from utils/src/core/ProcessContextBuilder.cpp
rename to core-framework/src/core/ProcessContextBuilder.cpp
diff --git a/utils/src/core/ProcessSession.cpp b/core-framework/src/core/ProcessSession.cpp
similarity index 100%
rename from utils/src/core/ProcessSession.cpp
rename to core-framework/src/core/ProcessSession.cpp
diff --git a/utils/src/core/ProcessorImpl.cpp b/core-framework/src/core/ProcessorImpl.cpp
similarity index 100%
rename from utils/src/core/ProcessorImpl.cpp
rename to core-framework/src/core/ProcessorImpl.cpp
diff --git a/utils/src/core/ProcessorMetrics.cpp b/core-framework/src/core/ProcessorMetrics.cpp
similarity index 100%
rename from utils/src/core/ProcessorMetrics.cpp
rename to core-framework/src/core/ProcessorMetrics.cpp
diff --git a/utils/src/core/Property.cpp b/core-framework/src/core/Property.cpp
similarity index 100%
rename from utils/src/core/Property.cpp
rename to core-framework/src/core/Property.cpp
diff --git a/utils/src/core/Repository.cpp b/core-framework/src/core/Repository.cpp
similarity index 100%
rename from utils/src/core/Repository.cpp
rename to core-framework/src/core/Repository.cpp
diff --git a/utils/src/core/TypedValues.cpp b/core-framework/src/core/TypedValues.cpp
similarity index 100%
rename from utils/src/core/TypedValues.cpp
rename to core-framework/src/core/TypedValues.cpp
diff --git a/utils/src/core/extension/Extension.cpp b/core-framework/src/core/extension/Extension.cpp
similarity index 100%
rename from utils/src/core/extension/Extension.cpp
rename to core-framework/src/core/extension/Extension.cpp
diff --git a/utils/src/http/BaseHTTPClient.cpp b/core-framework/src/http/BaseHTTPClient.cpp
similarity index 100%
rename from utils/src/http/BaseHTTPClient.cpp
rename to core-framework/src/http/BaseHTTPClient.cpp
diff --git a/utils/src/http/HTTPClient.cpp b/core-framework/src/http/HTTPClient.cpp
similarity index 100%
rename from utils/src/http/HTTPClient.cpp
rename to core-framework/src/http/HTTPClient.cpp
diff --git a/utils/src/http/HTTPStream.cpp b/core-framework/src/http/HTTPStream.cpp
similarity index 100%
rename from utils/src/http/HTTPStream.cpp
rename to core-framework/src/http/HTTPStream.cpp
diff --git a/utils/src/io/BufferStream.cpp b/core-framework/src/io/BufferStream.cpp
similarity index 100%
rename from utils/src/io/BufferStream.cpp
rename to core-framework/src/io/BufferStream.cpp
diff --git a/utils/src/io/CPPLINT.cfg b/core-framework/src/io/CPPLINT.cfg
similarity index 100%
rename from utils/src/io/CPPLINT.cfg
rename to core-framework/src/io/CPPLINT.cfg
diff --git a/utils/src/io/CRCStream.cpp b/core-framework/src/io/CRCStream.cpp
similarity index 100%
rename from utils/src/io/CRCStream.cpp
rename to core-framework/src/io/CRCStream.cpp
diff --git a/utils/src/io/FileStream.cpp b/core-framework/src/io/FileStream.cpp
similarity index 100%
rename from utils/src/io/FileStream.cpp
rename to core-framework/src/io/FileStream.cpp
diff --git a/utils/src/io/InputStream.cpp b/core-framework/src/io/InputStream.cpp
similarity index 100%
rename from utils/src/io/InputStream.cpp
rename to core-framework/src/io/InputStream.cpp
diff --git a/utils/src/io/OutputStream.cpp b/core-framework/src/io/OutputStream.cpp
similarity index 100%
rename from utils/src/io/OutputStream.cpp
rename to core-framework/src/io/OutputStream.cpp
diff --git a/utils/src/io/StreamSlice.cpp b/core-framework/src/io/StreamSlice.cpp
similarity index 100%
rename from utils/src/io/StreamSlice.cpp
rename to core-framework/src/io/StreamSlice.cpp
diff --git a/utils/src/io/ZlibStream.cpp b/core-framework/src/io/ZlibStream.cpp
similarity index 100%
rename from utils/src/io/ZlibStream.cpp
rename to core-framework/src/io/ZlibStream.cpp
diff --git a/utils/src/utils/AutoPersistor.cpp b/core-framework/src/utils/AutoPersistor.cpp
similarity index 100%
rename from utils/src/utils/AutoPersistor.cpp
rename to core-framework/src/utils/AutoPersistor.cpp
diff --git a/utils/src/utils/BackTrace.cpp b/core-framework/src/utils/BackTrace.cpp
similarity index 100%
rename from utils/src/utils/BackTrace.cpp
rename to core-framework/src/utils/BackTrace.cpp
diff --git a/utils/src/utils/ByteArrayCallback.cpp b/core-framework/src/utils/ByteArrayCallback.cpp
similarity index 100%
rename from utils/src/utils/ByteArrayCallback.cpp
rename to core-framework/src/utils/ByteArrayCallback.cpp
diff --git a/utils/src/utils/ClassUtils.cpp b/core-framework/src/utils/ClassUtils.cpp
similarity index 100%
rename from utils/src/utils/ClassUtils.cpp
rename to core-framework/src/utils/ClassUtils.cpp
diff --git a/utils/src/utils/ConfigurationUtils.cpp b/core-framework/src/utils/ConfigurationUtils.cpp
similarity index 100%
rename from utils/src/utils/ConfigurationUtils.cpp
rename to core-framework/src/utils/ConfigurationUtils.cpp
diff --git a/utils/src/utils/Cron.cpp b/core-framework/src/utils/Cron.cpp
similarity index 100%
rename from utils/src/utils/Cron.cpp
rename to core-framework/src/utils/Cron.cpp
diff --git a/utils/src/utils/Environment.cpp b/core-framework/src/utils/Environment.cpp
similarity index 100%
rename from utils/src/utils/Environment.cpp
rename to core-framework/src/utils/Environment.cpp
diff --git a/utils/src/utils/Error.cpp b/core-framework/src/utils/Error.cpp
similarity index 100%
rename from utils/src/utils/Error.cpp
rename to core-framework/src/utils/Error.cpp
diff --git a/utils/src/utils/FifoExecutor.cpp b/core-framework/src/utils/FifoExecutor.cpp
similarity index 100%
rename from utils/src/utils/FifoExecutor.cpp
rename to core-framework/src/utils/FifoExecutor.cpp
diff --git a/utils/src/utils/FileMutex.cpp b/core-framework/src/utils/FileMutex.cpp
similarity index 100%
rename from utils/src/utils/FileMutex.cpp
rename to core-framework/src/utils/FileMutex.cpp
diff --git a/utils/src/utils/HTTPUtils.cpp b/core-framework/src/utils/HTTPUtils.cpp
similarity index 100%
rename from utils/src/utils/HTTPUtils.cpp
rename to core-framework/src/utils/HTTPUtils.cpp
diff --git a/utils/src/utils/Id.cpp b/core-framework/src/utils/Id.cpp
similarity index 100%
rename from utils/src/utils/Id.cpp
rename to core-framework/src/utils/Id.cpp
diff --git a/utils/src/utils/LineByLineInputOutputStreamCallback.cpp b/core-framework/src/utils/LineByLineInputOutputStreamCallback.cpp
similarity index 100%
rename from utils/src/utils/LineByLineInputOutputStreamCallback.cpp
rename to core-framework/src/utils/LineByLineInputOutputStreamCallback.cpp
diff --git a/utils/src/utils/OsUtils.cpp b/core-framework/src/utils/OsUtils.cpp
similarity index 100%
rename from utils/src/utils/OsUtils.cpp
rename to core-framework/src/utils/OsUtils.cpp
diff --git a/utils/src/utils/ParsingUtils.cpp b/core-framework/src/utils/ParsingUtils.cpp
similarity index 100%
rename from utils/src/utils/ParsingUtils.cpp
rename to core-framework/src/utils/ParsingUtils.cpp
diff --git a/utils/src/utils/ProcessCpuUsageTracker.cpp b/core-framework/src/utils/ProcessCpuUsageTracker.cpp
similarity index 100%
rename from utils/src/utils/ProcessCpuUsageTracker.cpp
rename to core-framework/src/utils/ProcessCpuUsageTracker.cpp
diff --git a/utils/src/utils/RegexUtils.cpp b/core-framework/src/utils/RegexUtils.cpp
similarity index 100%
rename from utils/src/utils/RegexUtils.cpp
rename to core-framework/src/utils/RegexUtils.cpp
diff --git a/utils/src/utils/StoppableThread.cpp b/core-framework/src/utils/StoppableThread.cpp
similarity index 100%
rename from utils/src/utils/StoppableThread.cpp
rename to core-framework/src/utils/StoppableThread.cpp
diff --git a/utils/src/utils/StringUtils.cpp b/core-framework/src/utils/StringUtils.cpp
similarity index 100%
rename from utils/src/utils/StringUtils.cpp
rename to core-framework/src/utils/StringUtils.cpp
diff --git a/utils/src/utils/SystemCpuUsageTracker.cpp b/core-framework/src/utils/SystemCpuUsageTracker.cpp
similarity index 100%
rename from utils/src/utils/SystemCpuUsageTracker.cpp
rename to core-framework/src/utils/SystemCpuUsageTracker.cpp
diff --git a/utils/src/utils/ThreadPool.cpp b/core-framework/src/utils/ThreadPool.cpp
similarity index 100%
rename from utils/src/utils/ThreadPool.cpp
rename to core-framework/src/utils/ThreadPool.cpp
diff --git a/utils/src/utils/TimeUtil.cpp b/core-framework/src/utils/TimeUtil.cpp
similarity index 100%
rename from utils/src/utils/TimeUtil.cpp
rename to core-framework/src/utils/TimeUtil.cpp
diff --git a/utils/src/utils/crypto/EncryptionManager.cpp b/core-framework/src/utils/crypto/EncryptionManager.cpp
similarity index 100%
rename from utils/src/utils/crypto/EncryptionManager.cpp
rename to core-framework/src/utils/crypto/EncryptionManager.cpp
diff --git a/utils/src/utils/crypto/EncryptionProvider.cpp b/core-framework/src/utils/crypto/EncryptionProvider.cpp
similarity index 100%
rename from utils/src/utils/crypto/EncryptionProvider.cpp
rename to core-framework/src/utils/crypto/EncryptionProvider.cpp
diff --git a/utils/src/utils/crypto/EncryptionUtils.cpp b/core-framework/src/utils/crypto/EncryptionUtils.cpp
similarity index 100%
rename from utils/src/utils/crypto/EncryptionUtils.cpp
rename to core-framework/src/utils/crypto/EncryptionUtils.cpp
diff --git a/utils/src/utils/crypto/ciphers/Aes256Ecb.cpp b/core-framework/src/utils/crypto/ciphers/Aes256Ecb.cpp
similarity index 100%
rename from utils/src/utils/crypto/ciphers/Aes256Ecb.cpp
rename to core-framework/src/utils/crypto/ciphers/Aes256Ecb.cpp
diff --git a/utils/src/utils/crypto/property_encryption/PropertyEncryptionUtils.cpp b/core-framework/src/utils/crypto/property_encryption/PropertyEncryptionUtils.cpp
similarity index 100%
rename from utils/src/utils/crypto/property_encryption/PropertyEncryptionUtils.cpp
rename to core-framework/src/utils/crypto/property_encryption/PropertyEncryptionUtils.cpp
diff --git a/utils/src/utils/file/FilePattern.cpp b/core-framework/src/utils/file/FilePattern.cpp
similarity index 100%
rename from utils/src/utils/file/FilePattern.cpp
rename to core-framework/src/utils/file/FilePattern.cpp
diff --git a/utils/src/utils/file/FileUtils.cpp b/core-framework/src/utils/file/FileUtils.cpp
similarity index 100%
rename from utils/src/utils/file/FileUtils.cpp
rename to core-framework/src/utils/file/FileUtils.cpp
diff --git a/utils/src/utils/file/PathUtils.cpp b/core-framework/src/utils/file/PathUtils.cpp
similarity index 100%
rename from utils/src/utils/file/PathUtils.cpp
rename to core-framework/src/utils/file/PathUtils.cpp
diff --git a/utils/src/utils/net/AsioSocketUtils.cpp b/core-framework/src/utils/net/AsioSocketUtils.cpp
similarity index 100%
rename from utils/src/utils/net/AsioSocketUtils.cpp
rename to core-framework/src/utils/net/AsioSocketUtils.cpp
diff --git a/utils/src/utils/net/DNS.cpp b/core-framework/src/utils/net/DNS.cpp
similarity index 100%
rename from utils/src/utils/net/DNS.cpp
rename to core-framework/src/utils/net/DNS.cpp
diff --git a/utils/src/utils/net/NetworkInterfaceInfo.cpp b/core-framework/src/utils/net/NetworkInterfaceInfo.cpp
similarity index 100%
rename from utils/src/utils/net/NetworkInterfaceInfo.cpp
rename to core-framework/src/utils/net/NetworkInterfaceInfo.cpp
diff --git a/utils/src/utils/net/Socket.cpp b/core-framework/src/utils/net/Socket.cpp
similarity index 100%
rename from utils/src/utils/net/Socket.cpp
rename to core-framework/src/utils/net/Socket.cpp
diff --git a/extension-utils/CMakeLists.txt b/extension-framework/CMakeLists.txt
similarity index 56%
rename from extension-utils/CMakeLists.txt
rename to extension-framework/CMakeLists.txt
index f5c969d8f..8fbbe0ede 100644
--- a/extension-utils/CMakeLists.txt
+++ b/extension-framework/CMakeLists.txt
@@ -9,9 +9,9 @@ file(GLOB SOURCES
src/utils/net/*.cpp
src/utils/tls/*.cpp
src/utils/*.cpp)
-add_minifi_library(minifi-extension-utils STATIC ${SOURCES})
-target_include_directories(minifi-extension-utils PUBLIC include)
-target_link_libraries(minifi-extension-utils PUBLIC minifi-api)
+add_minifi_library(minifi-extension-framework STATIC ${SOURCES})
+target_include_directories(minifi-extension-framework PUBLIC include)
+target_link_libraries(minifi-extension-framework PUBLIC minifi-api)
include(RangeV3)
include(Asio)
@@ -23,13 +23,13 @@ endif()
if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
list(APPEND CORE_LIBRARIES stdc++fs)
endif()
-target_link_libraries(minifi-extension-utils PUBLIC ${CMAKE_DL_LIBS} ${CORE_LIBRARIES})
+target_link_libraries(minifi-extension-framework PUBLIC ${CMAKE_DL_LIBS} ${CORE_LIBRARIES})
if (APPLE)
- target_link_libraries(minifi-extension-utils PUBLIC "-framework CoreFoundation -framework SystemConfiguration")
+ target_link_libraries(minifi-extension-framework PUBLIC "-framework CoreFoundation -framework SystemConfiguration")
endif()
if (UNIX AND (CMAKE_SYSTEM_PROCESSOR MATCHES "armv7"))
- target_link_libraries(minifi-extension-utils PUBLIC "-latomic")
+ target_link_libraries(minifi-extension-framework PUBLIC "-latomic")
endif()
-target_link_libraries(minifi-extension-utils PRIVATE minifi-api)
-target_link_libraries(minifi-extension-utils PUBLIC minifi-utils)
+target_link_libraries(minifi-extension-framework PRIVATE minifi-api)
+target_link_libraries(minifi-extension-framework PUBLIC minifi-core-framework)
diff --git a/extension-utils/include/FlowFileRecord.h b/extension-framework/include/FlowFileRecord.h
similarity index 100%
rename from extension-utils/include/FlowFileRecord.h
rename to extension-framework/include/FlowFileRecord.h
diff --git a/extension-utils/include/ResourceClaim.h b/extension-framework/include/ResourceClaim.h
similarity index 100%
rename from extension-utils/include/ResourceClaim.h
rename to extension-framework/include/ResourceClaim.h
diff --git a/extension-utils/include/controllers/AttributeProviderService.h b/extension-framework/include/controllers/AttributeProviderService.h
similarity index 100%
rename from extension-utils/include/controllers/AttributeProviderService.h
rename to extension-framework/include/controllers/AttributeProviderService.h
diff --git a/extension-utils/include/controllers/RecordSetReader.h b/extension-framework/include/controllers/RecordSetReader.h
similarity index 100%
rename from extension-utils/include/controllers/RecordSetReader.h
rename to extension-framework/include/controllers/RecordSetReader.h
diff --git a/extension-utils/include/controllers/RecordSetWriter.h b/extension-framework/include/controllers/RecordSetWriter.h
similarity index 100%
rename from extension-utils/include/controllers/RecordSetWriter.h
rename to extension-framework/include/controllers/RecordSetWriter.h
diff --git a/extension-utils/include/controllers/SSLContextServiceInterface.h b/extension-framework/include/controllers/SSLContextService.h
similarity index 100%
copy from extension-utils/include/controllers/SSLContextServiceInterface.h
copy to extension-framework/include/controllers/SSLContextService.h
diff --git a/extension-utils/include/controllers/SSLContextServiceInterface.h b/extension-framework/include/controllers/SSLContextServiceInterface.h
similarity index 100%
rename from extension-utils/include/controllers/SSLContextServiceInterface.h
rename to extension-framework/include/controllers/SSLContextServiceInterface.h
diff --git a/extension-utils/include/controllers/keyvalue/KeyValueStateManager.h b/extension-framework/include/controllers/keyvalue/KeyValueStateManager.h
similarity index 100%
rename from extension-utils/include/controllers/keyvalue/KeyValueStateManager.h
rename to extension-framework/include/controllers/keyvalue/KeyValueStateManager.h
diff --git a/extension-utils/include/controllers/keyvalue/KeyValueStateStorage.h b/extension-framework/include/controllers/keyvalue/KeyValueStateStorage.h
similarity index 100%
rename from extension-utils/include/controllers/keyvalue/KeyValueStateStorage.h
rename to extension-framework/include/controllers/keyvalue/KeyValueStateStorage.h
diff --git a/extension-utils/include/core/AbstractProcessor.h b/extension-framework/include/core/AbstractProcessor.h
similarity index 100%
rename from extension-utils/include/core/AbstractProcessor.h
rename to extension-framework/include/core/AbstractProcessor.h
diff --git a/extension-utils/include/core/FlowFile.h b/extension-framework/include/core/FlowFile.h
similarity index 100%
rename from extension-utils/include/core/FlowFile.h
rename to extension-framework/include/core/FlowFile.h
diff --git a/extension-utils/include/core/FlowFileStore.h b/extension-framework/include/core/FlowFileStore.h
similarity index 100%
rename from extension-utils/include/core/FlowFileStore.h
rename to extension-framework/include/core/FlowFileStore.h
diff --git a/extension-utils/include/core/ProcessSession.h b/extension-framework/include/core/ProcessSession.h
similarity index 100%
rename from extension-utils/include/core/ProcessSession.h
rename to extension-framework/include/core/ProcessSession.h
diff --git a/extension-utils/include/core/StateManager.h b/extension-framework/include/core/StateManager.h
similarity index 100%
rename from extension-utils/include/core/StateManager.h
rename to extension-framework/include/core/StateManager.h
diff --git a/extension-utils/include/core/StateStorage.h b/extension-framework/include/core/StateStorage.h
similarity index 100%
rename from extension-utils/include/core/StateStorage.h
rename to extension-framework/include/core/StateStorage.h
diff --git a/extension-utils/include/properties/Configure.h b/extension-framework/include/properties/Configure.h
similarity index 100%
rename from extension-utils/include/properties/Configure.h
rename to extension-framework/include/properties/Configure.h
diff --git a/extension-utils/include/properties/Properties.h b/extension-framework/include/properties/Properties.h
similarity index 100%
rename from extension-utils/include/properties/Properties.h
rename to extension-framework/include/properties/Properties.h
diff --git a/extension-utils/include/serialization/FlowFileSerializer.h b/extension-framework/include/serialization/FlowFileSerializer.h
similarity index 100%
rename from extension-utils/include/serialization/FlowFileSerializer.h
rename to extension-framework/include/serialization/FlowFileSerializer.h
diff --git a/extension-utils/include/serialization/FlowFileV3Serializer.h b/extension-framework/include/serialization/FlowFileV3Serializer.h
similarity index 100%
rename from extension-utils/include/serialization/FlowFileV3Serializer.h
rename to extension-framework/include/serialization/FlowFileV3Serializer.h
diff --git a/extension-utils/include/serialization/PayloadSerializer.h b/extension-framework/include/serialization/PayloadSerializer.h
similarity index 100%
rename from extension-utils/include/serialization/PayloadSerializer.h
rename to extension-framework/include/serialization/PayloadSerializer.h
diff --git a/extension-utils/include/utils/ListingStateManager.h b/extension-framework/include/utils/ListingStateManager.h
similarity index 100%
rename from extension-utils/include/utils/ListingStateManager.h
rename to extension-framework/include/utils/ListingStateManager.h
diff --git a/extension-utils/include/utils/OpenTelemetryLogDataModelUtils.h b/extension-framework/include/utils/OpenTelemetryLogDataModelUtils.h
similarity index 100%
rename from extension-utils/include/utils/OpenTelemetryLogDataModelUtils.h
rename to extension-framework/include/utils/OpenTelemetryLogDataModelUtils.h
diff --git a/extension-utils/include/utils/ProcessorConfigUtils.h b/extension-framework/include/utils/ProcessorConfigUtils.h
similarity index 100%
rename from extension-utils/include/utils/ProcessorConfigUtils.h
rename to extension-framework/include/utils/ProcessorConfigUtils.h
diff --git a/extension-utils/include/utils/ResourceQueue.h b/extension-framework/include/utils/ResourceQueue.h
similarity index 100%
rename from extension-utils/include/utils/ResourceQueue.h
rename to extension-framework/include/utils/ResourceQueue.h
diff --git a/extension-utils/include/utils/file/FileManager.h b/extension-framework/include/utils/file/FileManager.h
similarity index 100%
rename from extension-utils/include/utils/file/FileManager.h
rename to extension-framework/include/utils/file/FileManager.h
diff --git a/extension-utils/include/utils/file/FileReaderCallback.h b/extension-framework/include/utils/file/FileReaderCallback.h
similarity index 100%
rename from extension-utils/include/utils/file/FileReaderCallback.h
rename to extension-framework/include/utils/file/FileReaderCallback.h
diff --git a/extension-utils/include/utils/file/FileWriterCallback.h b/extension-framework/include/utils/file/FileWriterCallback.h
similarity index 100%
rename from extension-utils/include/utils/file/FileWriterCallback.h
rename to extension-framework/include/utils/file/FileWriterCallback.h
diff --git a/extension-utils/include/utils/file/ListedFile.h b/extension-framework/include/utils/file/ListedFile.h
similarity index 100%
rename from extension-utils/include/utils/file/ListedFile.h
rename to extension-framework/include/utils/file/ListedFile.h
diff --git a/extension-utils/include/utils/net/ConnectionHandler.h b/extension-framework/include/utils/net/ConnectionHandler.h
similarity index 100%
rename from extension-utils/include/utils/net/ConnectionHandler.h
rename to extension-framework/include/utils/net/ConnectionHandler.h
diff --git a/extension-utils/include/utils/net/ConnectionHandlerBase.h b/extension-framework/include/utils/net/ConnectionHandlerBase.h
similarity index 100%
rename from extension-utils/include/utils/net/ConnectionHandlerBase.h
rename to extension-framework/include/utils/net/ConnectionHandlerBase.h
diff --git a/extension-utils/include/utils/net/IpProtocol.h b/extension-framework/include/utils/net/IpProtocol.h
similarity index 100%
rename from extension-utils/include/utils/net/IpProtocol.h
rename to extension-framework/include/utils/net/IpProtocol.h
diff --git a/extension-utils/include/utils/net/Message.h b/extension-framework/include/utils/net/Message.h
similarity index 100%
rename from extension-utils/include/utils/net/Message.h
rename to extension-framework/include/utils/net/Message.h
diff --git a/extension-utils/include/utils/net/Server.h b/extension-framework/include/utils/net/Server.h
similarity index 100%
rename from extension-utils/include/utils/net/Server.h
rename to extension-framework/include/utils/net/Server.h
diff --git a/extension-utils/include/utils/net/Ssl.h b/extension-framework/include/utils/net/Ssl.h
similarity index 100%
rename from extension-utils/include/utils/net/Ssl.h
rename to extension-framework/include/utils/net/Ssl.h
diff --git a/extension-utils/include/utils/net/TcpServer.h b/extension-framework/include/utils/net/TcpServer.h
similarity index 100%
rename from extension-utils/include/utils/net/TcpServer.h
rename to extension-framework/include/utils/net/TcpServer.h
diff --git a/extension-utils/include/utils/net/UdpServer.h b/extension-framework/include/utils/net/UdpServer.h
similarity index 100%
rename from extension-utils/include/utils/net/UdpServer.h
rename to extension-framework/include/utils/net/UdpServer.h
diff --git a/extension-utils/src/controllers/keyvalue/KeyValueStateManager.cpp b/extension-framework/src/controllers/keyvalue/KeyValueStateManager.cpp
similarity index 100%
rename from extension-utils/src/controllers/keyvalue/KeyValueStateManager.cpp
rename to extension-framework/src/controllers/keyvalue/KeyValueStateManager.cpp
diff --git a/extension-utils/src/controllers/keyvalue/KeyValueStateStorage.cpp b/extension-framework/src/controllers/keyvalue/KeyValueStateStorage.cpp
similarity index 100%
rename from extension-utils/src/controllers/keyvalue/KeyValueStateStorage.cpp
rename to extension-framework/src/controllers/keyvalue/KeyValueStateStorage.cpp
diff --git a/extension-utils/src/core/ProcessContext.cpp b/extension-framework/src/core/ProcessContext.cpp
similarity index 100%
rename from extension-utils/src/core/ProcessContext.cpp
rename to extension-framework/src/core/ProcessContext.cpp
diff --git a/extension-utils/src/serialization/FlowFileV3Serializer.cpp b/extension-framework/src/serialization/FlowFileV3Serializer.cpp
similarity index 100%
rename from extension-utils/src/serialization/FlowFileV3Serializer.cpp
rename to extension-framework/src/serialization/FlowFileV3Serializer.cpp
diff --git a/extension-utils/src/serialization/PayloadSerializer.cpp b/extension-framework/src/serialization/PayloadSerializer.cpp
similarity index 100%
rename from extension-utils/src/serialization/PayloadSerializer.cpp
rename to extension-framework/src/serialization/PayloadSerializer.cpp
diff --git a/extension-utils/src/utils/CallBackTimer.cpp b/extension-framework/src/utils/CallBackTimer.cpp
similarity index 100%
rename from extension-utils/src/utils/CallBackTimer.cpp
rename to extension-framework/src/utils/CallBackTimer.cpp
diff --git a/extension-utils/src/utils/ListingStateManager.cpp b/extension-framework/src/utils/ListingStateManager.cpp
similarity index 100%
rename from extension-utils/src/utils/ListingStateManager.cpp
rename to extension-framework/src/utils/ListingStateManager.cpp
diff --git a/extension-utils/src/utils/file/FileReaderCallback.cpp b/extension-framework/src/utils/file/FileReaderCallback.cpp
similarity index 100%
rename from extension-utils/src/utils/file/FileReaderCallback.cpp
rename to extension-framework/src/utils/file/FileReaderCallback.cpp
diff --git a/extension-utils/src/utils/file/FileWriterCallback.cpp b/extension-framework/src/utils/file/FileWriterCallback.cpp
similarity index 100%
rename from extension-utils/src/utils/file/FileWriterCallback.cpp
rename to extension-framework/src/utils/file/FileWriterCallback.cpp
diff --git a/extension-utils/src/utils/net/Ssl.cpp b/extension-framework/src/utils/net/Ssl.cpp
similarity index 100%
rename from extension-utils/src/utils/net/Ssl.cpp
rename to extension-framework/src/utils/net/Ssl.cpp
diff --git a/extension-utils/src/utils/net/TcpServer.cpp b/extension-framework/src/utils/net/TcpServer.cpp
similarity index 100%
rename from extension-utils/src/utils/net/TcpServer.cpp
rename to extension-framework/src/utils/net/TcpServer.cpp
diff --git a/extension-utils/src/utils/net/UdpServer.cpp b/extension-framework/src/utils/net/UdpServer.cpp
similarity index 100%
rename from extension-utils/src/utils/net/UdpServer.cpp
rename to extension-framework/src/utils/net/UdpServer.cpp
diff --git a/extension-utils/include/controllers/SSLContextService.h b/extension-utils/include/controllers/SSLContextService.h
new file mode 100644
index 000000000..e69de29bb
diff --git a/extensions/rocksdb-repos/CMakeLists.txt b/extensions/rocksdb-repos/CMakeLists.txt
index c5e507113..f4d535208 100644
--- a/extensions/rocksdb-repos/CMakeLists.txt
+++ b/extensions/rocksdb-repos/CMakeLists.txt
@@ -31,7 +31,7 @@ file(GLOB SOURCES "*.cpp" "controllers/*.cpp" "database/*.cpp" "encryption/*.cp
add_minifi_library(minifi-rocksdb-repos SHARED ${SOURCES})
target_link_libraries(minifi-rocksdb-repos PUBLIC RocksDB::RocksDB)
-target_link_libraries(minifi-rocksdb-repos PUBLIC minifi-api minifi-extension-utils Threads::Threads)
+target_link_libraries(minifi-rocksdb-repos PUBLIC minifi-api minifi-extension-framework Threads::Threads)
target_link_libraries(minifi-rocksdb-repos PRIVATE $<LINK_ONLY:core-minifi>)
register_extension(minifi-rocksdb-repos "ROCKSDB REPOS" ROCKSDB-REPOS "This Enables persistent provenance, flowfile, and content repositories using RocksDB" "extensions/rocksdb-repos/tests")
diff --git a/libminifi/CMakeLists.txt b/libminifi/CMakeLists.txt
index 116a41a0c..80c2f4624 100644
--- a/libminifi/CMakeLists.txt
+++ b/libminifi/CMakeLists.txt
@@ -84,7 +84,7 @@ endif()
include(RangeV3)
include(Asio)
include(MagicEnum)
-list(APPEND LIBMINIFI_LIBRARIES minifi-utils yaml-cpp ZLIB::ZLIB concurrentqueue RapidJSON spdlog::spdlog Threads::Threads gsl-lite libsodium range-v3 expected-lite asio magic_enum OpenSSL::Crypto OpenSSL::SSL CURL::libcurl RapidJSON fmt::fmt)
+list(APPEND LIBMINIFI_LIBRARIES minifi-core-framework yaml-cpp ZLIB::ZLIB concurrentqueue RapidJSON spdlog::spdlog Threads::Threads gsl-lite libsodium range-v3 expected-lite asio magic_enum OpenSSL::Crypto OpenSSL::SSL CURL::libcurl RapidJSON fmt::fmt)
if(NOT WIN32)
list(APPEND LIBMINIFI_LIBRARIES OSSP::libuuid++)
endif()
@@ -99,7 +99,7 @@ if (UNIX AND (CMAKE_SYSTEM_PROCESSOR MATCHES "armv7"))
target_link_libraries(core-minifi "-latomic")
endif()
-SET(LIBMINIFI minifi-api minifi-extension-utils $<LINK_ONLY:core-minifi> PARENT_SCOPE)
+SET(LIBMINIFI minifi-api minifi-extension-framework $<LINK_ONLY:core-minifi> PARENT_SCOPE)
if (WIN32)
install(TARGETS core-minifi ARCHIVE DESTINATION bin COMPONENT bin)
diff --git a/libminifi/test/libtest/unit/CMakeLists.txt b/libminifi/test/libtest/unit/CMakeLists.txt
index 1a6220fe8..674d1969b 100644
--- a/libminifi/test/libtest/unit/CMakeLists.txt
+++ b/libminifi/test/libtest/unit/CMakeLists.txt
@@ -18,7 +18,7 @@
#
file(GLOB TEST_BASE_SOURCES "*.cpp")
add_minifi_library(libminifi-unittest STATIC ${TEST_BASE_SOURCES})
-target_link_libraries(libminifi-unittest core-minifi minifi-extension-utils Catch2WithMain)
+target_link_libraries(libminifi-unittest core-minifi minifi-extension-framework Catch2WithMain)
target_include_directories(libminifi-unittest BEFORE PRIVATE "${CMAKE_SOURCE_DIR}/libminifi/include/")
if(WIN32)
target_include_directories(libminifi-unittest BEFORE PRIVATE "${CMAKE_SOURCE_DIR}/libminifi/opsys/win")
diff --git a/utils/CMakeLists.txt b/utils/CMakeLists.txt
deleted file mode 100644
index b789969a4..000000000
--- a/utils/CMakeLists.txt
+++ /dev/null
@@ -1,22 +0,0 @@
-file(GLOB SOURCES
- src/*.cpp
- src/core/*.cpp
- src/core/extension/*.cpp
- src/io/*.cpp
- src/http/*.cpp
- src/utils/*.cpp
- src/utils/crypto/*.cpp
- src/utils/crypto/ciphers/*.cpp
- src/utils/crypto/property_encryption/*.cpp
- src/utils/net/*.cpp
- src/utils/file/*.cpp)
-
-add_minifi_library(minifi-utils STATIC ${SOURCES})
-target_include_directories(minifi-utils PUBLIC include)
-target_link_libraries(minifi-utils PUBLIC minifi-api ZLIB::ZLIB concurrentqueue RapidJSON spdlog Threads::Threads gsl-lite libsodium range-v3 expected-lite date::date date::tz asio magic_enum OpenSSL::Crypto OpenSSL::SSL CURL::libcurl RapidJSON)
-if(NOT WIN32)
- target_link_libraries(minifi-utils PUBLIC OSSP::libuuid++)
-endif()
-if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
- target_link_libraries(minifi-utils PUBLIC stdc++fs)
-endif()
(nifi-minifi-cpp) 35/37: Linter fix, rollback temp test changes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit ce6080e8e358692e12e678c68acb56231ba1af36
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Wed Jul 23 11:37:45 2025 +0200
Linter fix, rollback temp test changes
---
extensions/python/PythonProcessor.cpp | 2 -
extensions/sftp/processors/PutSFTP.cpp | 2 +-
extensions/sftp/processors/SFTPProcessorBase.cpp | 2 +-
.../standard-processors/modbus/FetchModbusTcp.cpp | 2 +-
.../standard-processors/modbus/FetchModbusTcp.h | 2 +-
.../tests/unit/DefragmentTextTests.cpp | 2 +-
libminifi/src/core/ProcessContext.cpp | 10 ++---
libminifi/test/integration/C2MetricsTest.cpp | 47 ++++++++++------------
libminifi/test/libtest/unit/TestBase.h | 5 +++
minifi-api/include/minifi-cpp/core/ClassLoader.h | 3 +-
.../include/minifi-cpp/core/ProcessContext.h | 10 ++---
11 files changed, 44 insertions(+), 43 deletions(-)
diff --git a/extensions/python/PythonProcessor.cpp b/extensions/python/PythonProcessor.cpp
index 2416b3288..c7a231f21 100644
--- a/extensions/python/PythonProcessor.cpp
+++ b/extensions/python/PythonProcessor.cpp
@@ -22,8 +22,6 @@
namespace org::apache::nifi::minifi::extensions::python {
-namespace core = org::apache::nifi::minifi::core;
-
PythonProcessor::PythonProcessor(python::processors::ExecutePythonProcessor* proc) :
processor_(proc) {
gsl_Expects(processor_);
diff --git a/extensions/sftp/processors/PutSFTP.cpp b/extensions/sftp/processors/PutSFTP.cpp
index 26a6d4bf7..41cd0b8bf 100644
--- a/extensions/sftp/processors/PutSFTP.cpp
+++ b/extensions/sftp/processors/PutSFTP.cpp
@@ -43,7 +43,7 @@ void PutSFTP::initialize() {
}
PutSFTP::PutSFTP(core::ProcessorMetadata metadata)
- : SFTPProcessorBase(metadata),
+ : SFTPProcessorBase(std::move(metadata)),
create_directory_(false),
batch_size_(0),
reject_zero_byte_(false),
diff --git a/extensions/sftp/processors/SFTPProcessorBase.cpp b/extensions/sftp/processors/SFTPProcessorBase.cpp
index a54fad7d2..4587d94ef 100644
--- a/extensions/sftp/processors/SFTPProcessorBase.cpp
+++ b/extensions/sftp/processors/SFTPProcessorBase.cpp
@@ -44,7 +44,7 @@
namespace org::apache::nifi::minifi::processors {
SFTPProcessorBase::SFTPProcessorBase(core::ProcessorMetadata metadata)
- : ProcessorImpl(metadata),
+ : ProcessorImpl(std::move(metadata)),
connection_timeout_(0),
data_timeout_(0),
strict_host_checking_(false),
diff --git a/extensions/standard-processors/modbus/FetchModbusTcp.cpp b/extensions/standard-processors/modbus/FetchModbusTcp.cpp
index dccdad866..eacea365b 100644
--- a/extensions/standard-processors/modbus/FetchModbusTcp.cpp
+++ b/extensions/standard-processors/modbus/FetchModbusTcp.cpp
@@ -117,7 +117,7 @@ void FetchModbusTcp::initialize() {
setSupportedRelationships(Relationships);
}
-std::shared_ptr<core::FlowFile> FetchModbusTcp::getOrCreateFlowFile(core::ProcessContext& context, core::ProcessSession& session) const {
+std::shared_ptr<core::FlowFile> FetchModbusTcp::getOrCreateFlowFile(core::ProcessContext& context, core::ProcessSession& session) {
if (context.hasIncomingConnections()) {
return session.get();
}
diff --git a/extensions/standard-processors/modbus/FetchModbusTcp.h b/extensions/standard-processors/modbus/FetchModbusTcp.h
index 9fb0c4d4a..e7ce18ef6 100644
--- a/extensions/standard-processors/modbus/FetchModbusTcp.h
+++ b/extensions/standard-processors/modbus/FetchModbusTcp.h
@@ -124,7 +124,7 @@ class FetchModbusTcp final : public core::ProcessorImpl {
asio::awaitable<nonstd::expected<core::RecordField, std::error_code>> sendRequestAndReadResponse(utils::net::ConnectionHandlerBase& connection_handler,
const ReadModbusFunction& read_modbus_function);
std::unordered_map<std::string, std::unique_ptr<ReadModbusFunction>> getAddressMap(core::ProcessContext& context, const core::FlowFile& flow_file);
- std::shared_ptr<core::FlowFile> getOrCreateFlowFile(core::ProcessContext& context, core::ProcessSession& session) const;
+ static std::shared_ptr<core::FlowFile> getOrCreateFlowFile(core::ProcessContext& context, core::ProcessSession& session);
void removeExpiredConnections();
asio::io_context io_context_;
diff --git a/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp b/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
index 28f1579f0..017c2d1b9 100644
--- a/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
+++ b/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
@@ -324,7 +324,7 @@ class FragmentGenerator : public core::ProcessorImpl {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
explicit FragmentGenerator(minifi::core::ProcessorMetadata metadata)
- : ProcessorImpl(metadata) {
+ : ProcessorImpl(std::move(metadata)) {
}
void onTrigger(core::ProcessContext&, core::ProcessSession& session) override {
diff --git a/libminifi/src/core/ProcessContext.cpp b/libminifi/src/core/ProcessContext.cpp
index 985a23271..3c985c9bc 100644
--- a/libminifi/src/core/ProcessContext.cpp
+++ b/libminifi/src/core/ProcessContext.cpp
@@ -26,11 +26,11 @@ class StandardProcessorInfo : public ProcessorInfo {
public:
explicit StandardProcessorInfo(Processor& proc): proc_(proc) {}
- std::string getName() const override {return proc_.getName();}
- utils::Identifier getUUID() const override {return proc_.getUUID();}
- std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const override {return proc_.getFlowIdentifier();}
- std::map<std::string, core::Property, std::less<>> getSupportedProperties() const override {return proc_.getSupportedProperties();}
- nonstd::expected<Property, std::error_code> getSupportedProperty(std::string_view name) const override {return proc_.getSupportedProperty(name);}
+ [[nodiscard]] std::string getName() const override {return proc_.getName();}
+ [[nodiscard]] utils::Identifier getUUID() const override {return proc_.getUUID();}
+ [[nodiscard]] std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const override {return proc_.getFlowIdentifier();}
+ [[nodiscard]] std::map<std::string, core::Property, std::less<>> getSupportedProperties() const override {return proc_.getSupportedProperties();}
+ [[nodiscard]] nonstd::expected<Property, std::error_code> getSupportedProperty(std::string_view name) const override {return proc_.getSupportedProperty(name);}
private:
Processor& proc_;
diff --git a/libminifi/test/integration/C2MetricsTest.cpp b/libminifi/test/integration/C2MetricsTest.cpp
index 75519215b..ac82324cf 100644
--- a/libminifi/test/integration/C2MetricsTest.cpp
+++ b/libminifi/test/integration/C2MetricsTest.cpp
@@ -58,8 +58,6 @@ class VerifyC2Metrics : public VerifyC2Base {
const std::atomic_bool& metrics_updated_successfully_;
};
-#define CHECK_EXPR(expr) (expr ? true : (std::cout << #expr << " failed" << std::endl, false))
-
class MetricsHandler: public HeartbeatHandler {
public:
explicit MetricsHandler(std::atomic_bool& metrics_updated_successfully, std::shared_ptr<minifi::Configure> configuration, const std::filesystem::path& replacement_config_path)
@@ -107,13 +105,13 @@ class MetricsHandler: public HeartbeatHandler {
void verifyMetrics(const rapidjson::Document& root) {
auto initial_metrics_verified =
- CHECK_EXPR(root.HasMember("metrics")) &&
- CHECK_EXPR(root["metrics"].HasMember("RuntimeMetrics")) &&
- CHECK_EXPR(root["metrics"].HasMember("LoadMetrics")) &&
- CHECK_EXPR(root["metrics"].HasMember("ProcessorMetrics")) &&
- CHECK_EXPR(verifyRuntimeMetrics(root["metrics"]["RuntimeMetrics"])) &&
- CHECK_EXPR(verifyLoadMetrics(root["metrics"]["LoadMetrics"])) &&
- CHECK_EXPR(verifyProcessorMetrics(root["metrics"]["ProcessorMetrics"]));
+ root.HasMember("metrics") &&
+ root["metrics"].HasMember("RuntimeMetrics") &&
+ root["metrics"].HasMember("LoadMetrics") &&
+ root["metrics"].HasMember("ProcessorMetrics") &&
+ verifyRuntimeMetrics(root["metrics"]["RuntimeMetrics"]) &&
+ verifyLoadMetrics(root["metrics"]["LoadMetrics"]) &&
+ verifyProcessorMetrics(root["metrics"]["ProcessorMetrics"]);
if (initial_metrics_verified) {
test_state_ = TestState::SEND_NEW_CONFIG;
}
@@ -165,42 +163,41 @@ class MetricsHandler: public HeartbeatHandler {
static bool verifyProcessorBulletins(const rapidjson::Value& runtime_metrics) {
if (!runtime_metrics["flowInfo"].HasMember("processorBulletins")) {
- std::cout << "!runtime_metrics[\"flowInfo\"].HasMember(\"processorBulletins\")" << std::endl;
+ std::cout << R"(!runtime_metrics["flowInfo"].HasMember("processorBulletins"))" << std::endl;
return false;
}
auto bulletins = runtime_metrics["flowInfo"]["processorBulletins"].GetArray();
return std::any_of(bulletins.begin(), bulletins.end(), [](const auto& bulletin) {
std::string message = bulletin["message"].GetString();
- return CHECK_EXPR(bulletin["id"].GetInt() > 0) &&
- CHECK_EXPR(bulletin["timestamp"].GetInt64() > 0) &&
- CHECK_EXPR(bulletin["level"].GetString() == std::string("ERROR")) &&
- CHECK_EXPR(bulletin["category"].GetString() == std::string("Log Message")) &&
- CHECK_EXPR(message.find("Error connecting to") != std::string::npos) &&
- CHECK_EXPR(message.find(GETTCP_UUID) != std::string::npos) &&
- CHECK_EXPR(bulletin["groupId"].GetString() == std::string(PROCESS_GROUP_UUID)) &&
- CHECK_EXPR(bulletin["groupName"].GetString() == std::string("MiNiFi Flow")) &&
- CHECK_EXPR(bulletin["groupPath"].GetString() == std::string("MiNiFi Flow")) &&
- CHECK_EXPR(bulletin["sourceId"].GetString() == std::string(GETTCP_UUID)) &&
- CHECK_EXPR(bulletin["sourceName"].GetString() == std::string("GetTCP"));
+ return bulletin["id"].GetInt() > 0 &&
+ bulletin["timestamp"].GetInt64() > 0 &&
+ bulletin["level"].GetString() == std::string("ERROR") &&
+ bulletin["category"].GetString() == std::string("Log Message") &&
+ message.find("Error connecting to") != std::string::npos &&
+ message.find(GETTCP_UUID) != std::string::npos &&
+ bulletin["groupId"].GetString() == std::string(PROCESS_GROUP_UUID) &&
+ bulletin["groupName"].GetString() == std::string("MiNiFi Flow") &&
+ bulletin["groupPath"].GetString() == std::string("MiNiFi Flow") &&
+ bulletin["sourceId"].GetString() == std::string(GETTCP_UUID) &&
+ bulletin["sourceName"].GetString() == std::string("GetTCP");
});
}
static bool verifyRuntimeMetrics(const rapidjson::Value& runtime_metrics) {
- return CHECK_EXPR(verifyCommonRuntimeMetricNodes(runtime_metrics, "2438e3c8-015a-1000-79ca-83af40ec1997")) &&
+ return verifyCommonRuntimeMetricNodes(runtime_metrics, "2438e3c8-015a-1000-79ca-83af40ec1997") &&
[&]() {
const auto processor_statuses = runtime_metrics["flowInfo"]["processorStatuses"].GetArray();
if (processor_statuses.Size() != 2) {
- std::cout << "processor_statuses.Size() != 2" << std::endl;
return false;
}
return std::all_of(processor_statuses.begin(), processor_statuses.end(), [&](const auto& processor) {
if (processor["id"].GetString() != std::string(GETTCP_UUID) && processor["id"].GetString() != std::string(LOGATTRIBUTE1_UUID)) {
throw std::runtime_error(std::string("Unexpected processor id in processorStatuses: ") + processor["id"].GetString());
}
- return CHECK_EXPR(processorMetricsAreValid(processor));
+ return processorMetricsAreValid(processor);
});
}() &&
- CHECK_EXPR(verifyProcessorBulletins(runtime_metrics));
+ verifyProcessorBulletins(runtime_metrics);
}
static bool verifyUpdatedRuntimeMetrics(const rapidjson::Value& runtime_metrics) {
diff --git a/libminifi/test/libtest/unit/TestBase.h b/libminifi/test/libtest/unit/TestBase.h
index 39b2856af..68759e462 100644
--- a/libminifi/test/libtest/unit/TestBase.h
+++ b/libminifi/test/libtest/unit/TestBase.h
@@ -231,6 +231,11 @@ class TypedProcessorWrapper {
return proc_;
}
+ core::Processor& operator*() const {
+ gsl_Assert(proc_);
+ return *proc_;
+ }
+
private:
core::Processor* proc_{nullptr};
};
diff --git a/minifi-api/include/minifi-cpp/core/ClassLoader.h b/minifi-api/include/minifi-cpp/core/ClassLoader.h
index 56e031f39..5413e74e5 100644
--- a/minifi-api/include/minifi-cpp/core/ClassLoader.h
+++ b/minifi-api/include/minifi-cpp/core/ClassLoader.h
@@ -75,7 +75,8 @@ class ClassLoader {
[[nodiscard]] virtual std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) = 0;
- [[nodiscard]] virtual std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const std::string &name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) = 0;
+ [[nodiscard]] virtual std::unique_ptr<CoreComponent> instantiate(
+ const std::string &class_name, const std::string &name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) = 0;
[[nodiscard]] virtual CoreComponent* instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) = 0;
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContext.h b/minifi-api/include/minifi-cpp/core/ProcessContext.h
index 1e1e3e2b0..446d104a0 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContext.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContext.h
@@ -34,11 +34,11 @@ namespace org::apache::nifi::minifi::core {
class ProcessorInfo {
public:
- virtual std::string getName() const = 0;
- virtual utils::Identifier getUUID() const = 0;
- virtual std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const = 0;
- virtual std::map<std::string, core::Property, std::less<>> getSupportedProperties() const = 0;
- virtual nonstd::expected<Property, std::error_code> getSupportedProperty(std::string_view name) const = 0;
+ [[nodiscard]] virtual std::string getName() const = 0;
+ [[nodiscard]] virtual utils::Identifier getUUID() const = 0;
+ [[nodiscard]] virtual std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const = 0;
+ [[nodiscard]] virtual std::map<std::string, core::Property, std::less<>> getSupportedProperties() const = 0;
+ [[nodiscard]] virtual nonstd::expected<Property, std::error_code> getSupportedProperty(std::string_view name) const = 0;
virtual ~ProcessorInfo() = default;
};
(nifi-minifi-cpp) 19/37: Fix llamacpp proc/test
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 13bda7c0dd50b030f12896650160b796df788be2
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Jun 23 10:53:15 2025 +0200
Fix llamacpp proc/test
---
.../llamacpp/processors/RunLlamaCppInference.h | 9 ++++---
.../llamacpp/tests/RunLlamaCppInferenceTests.cpp | 28 +++++++++++++++-------
2 files changed, 23 insertions(+), 14 deletions(-)
diff --git a/extensions/llamacpp/processors/RunLlamaCppInference.h b/extensions/llamacpp/processors/RunLlamaCppInference.h
index a9fd90a02..13eafa4e6 100644
--- a/extensions/llamacpp/processors/RunLlamaCppInference.h
+++ b/extensions/llamacpp/processors/RunLlamaCppInference.h
@@ -20,7 +20,7 @@
#include <mutex>
#include <atomic>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/logging/LoggerFactory.h"
#include "core/PropertyDefinitionBuilder.h"
#include "LlamaContext.h"
@@ -33,7 +33,7 @@ using LlamaContextProvider =
class RunLlamaCppInferenceMetrics : public core::ProcessorMetricsImpl {
public:
- explicit RunLlamaCppInferenceMetrics(const core::Processor& source_processor)
+ explicit RunLlamaCppInferenceMetrics(const core::ProcessorImpl& source_processor)
: core::ProcessorMetricsImpl(source_processor) {
}
@@ -63,8 +63,8 @@ class RunLlamaCppInferenceMetrics : public core::ProcessorMetricsImpl {
class RunLlamaCppInference : public core::ProcessorImpl {
public:
- explicit RunLlamaCppInference(std::string_view name, const utils::Identifier& uuid = {}, LlamaContextProvider llama_context_provider = {})
- : core::ProcessorImpl(name, uuid),
+ explicit RunLlamaCppInference(core::ProcessorMetadata info, LlamaContextProvider llama_context_provider = {})
+ : core::ProcessorImpl(info),
llama_context_provider_(std::move(llama_context_provider)) {
metrics_ = gsl::make_not_null(std::make_shared<RunLlamaCppInferenceMetrics>(*this));
}
@@ -186,7 +186,6 @@ class RunLlamaCppInference : public core::ProcessorImpl {
private:
void increaseTokensIn(uint64_t token_count);
void increaseTokensOut(uint64_t token_count);
- std::shared_ptr<core::logging::Logger> logger_ = core::logging::LoggerFactory<RunLlamaCppInference>::getLogger(uuid_);
std::string model_path_;
std::string system_prompt_;
diff --git a/extensions/llamacpp/tests/RunLlamaCppInferenceTests.cpp b/extensions/llamacpp/tests/RunLlamaCppInferenceTests.cpp
index bd514a244..cd57f574d 100644
--- a/extensions/llamacpp/tests/RunLlamaCppInferenceTests.cpp
+++ b/extensions/llamacpp/tests/RunLlamaCppInferenceTests.cpp
@@ -20,6 +20,7 @@
#include "RunLlamaCppInference.h"
#include "unit/SingleProcessorTestController.h"
#include "core/FlowFile.h"
+#include "unit/ProcessorUtils.h"
namespace org::apache::nifi::minifi::extensions::llamacpp::test {
@@ -78,7 +79,8 @@ TEST_CASE("Prompt is generated correctly with default parameters") {
std::filesystem::path test_model_path;
processors::LlamaSamplerParams test_sampler_params;
processors::LlamaContextParams test_context_params;
- minifi::test::SingleProcessorTestController controller(std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path& model_path, const processors::LlamaSamplerParams& sampler_params, const processors::LlamaContextParams& context_params) {
test_model_path = model_path;
test_sampler_params = sampler_params;
@@ -123,7 +125,8 @@ TEST_CASE("Prompt is generated correctly with custom parameters") {
std::filesystem::path test_model_path;
processors::LlamaSamplerParams test_sampler_params;
processors::LlamaContextParams test_context_params;
- minifi::test::SingleProcessorTestController controller(std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path& model_path, const processors::LlamaSamplerParams& sampler_params, const processors::LlamaContextParams& context_params) {
test_model_path = model_path;
test_sampler_params = sampler_params;
@@ -174,7 +177,8 @@ TEST_CASE("Prompt is generated correctly with custom parameters") {
TEST_CASE("Empty flow file does not include input data in prompt") {
auto mock_llama_context = std::make_unique<MockLlamaContext>();
auto mock_llama_context_ptr = mock_llama_context.get();
- minifi::test::SingleProcessorTestController controller(std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path&, const processors::LlamaSamplerParams&, const processors::LlamaContextParams&) {
return std::move(mock_llama_context);
}));
@@ -197,7 +201,8 @@ TEST_CASE("Empty flow file does not include input data in prompt") {
}
TEST_CASE("Invalid values for optional double type properties throw exception") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path&, const processors::LlamaSamplerParams&, const processors::LlamaContextParams&) {
return std::make_unique<MockLlamaContext>();
}));
@@ -225,7 +230,8 @@ TEST_CASE("Invalid values for optional double type properties throw exception")
TEST_CASE("Top K property empty and invalid values are handled properly") {
std::optional<int32_t> test_top_k = 0;
- minifi::test::SingleProcessorTestController controller(std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path&, const processors::LlamaSamplerParams& sampler_params, const processors::LlamaContextParams&) {
test_top_k = sampler_params.top_k;
return std::make_unique<MockLlamaContext>();
@@ -256,7 +262,8 @@ TEST_CASE("Error handling during generation and applying template") {
mock_llama_context->setApplyTemplateFailure();
}
- minifi::test::SingleProcessorTestController controller(std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path&, const processors::LlamaSamplerParams&, const processors::LlamaContextParams&) {
return std::move(mock_llama_context);
}));
@@ -273,7 +280,8 @@ TEST_CASE("Error handling during generation and applying template") {
}
TEST_CASE("Route flow file to failure when prompt and input data is empty") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path&, const processors::LlamaSamplerParams&, const processors::LlamaContextParams&) {
return std::make_unique<MockLlamaContext>();
}));
@@ -292,7 +300,8 @@ TEST_CASE("Route flow file to failure when prompt and input data is empty") {
TEST_CASE("System prompt is optional") {
auto mock_llama_context = std::make_unique<MockLlamaContext>();
auto mock_llama_context_ptr = mock_llama_context.get();
- minifi::test::SingleProcessorTestController controller(std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path&, const processors::LlamaSamplerParams&, const processors::LlamaContextParams&) {
return std::move(mock_llama_context);
}));
@@ -313,7 +322,8 @@ TEST_CASE("System prompt is optional") {
}
TEST_CASE("Test output metrics") {
- auto processor = std::make_unique<processors::RunLlamaCppInference>("RunLlamaCppInference", utils::Identifier(),
+ auto processor = minifi::test::utils::make_custom_processor<processors::RunLlamaCppInference>(
+ core::ProcessorMetadata{utils::Identifier{}, "RunLlamaCppInference", logging::LoggerFactory<processors::RunLlamaCppInference>::getLogger()},
[&](const std::filesystem::path&, const processors::LlamaSamplerParams&, const processors::LlamaContextParams&) {
return std::make_unique<MockLlamaContext>();
});
(nifi-minifi-cpp) 09/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit b208d843c13729346572d32b210dedc0e3ba5d7e
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Apr 28 10:28:42 2025 +0200
Rebase fix
---
extensions/systemd/ConsumeJournald.cpp | 10 ++++------
extensions/systemd/ConsumeJournald.h | 3 +--
extensions/systemd/tests/ConsumeJournaldTest.cpp | 4 ++--
3 files changed, 7 insertions(+), 10 deletions(-)
diff --git a/extensions/systemd/ConsumeJournald.cpp b/extensions/systemd/ConsumeJournald.cpp
index fa212bad7..ca0cfa276 100644
--- a/extensions/systemd/ConsumeJournald.cpp
+++ b/extensions/systemd/ConsumeJournald.cpp
@@ -33,10 +33,8 @@ namespace org::apache::nifi::minifi::extensions::systemd {
namespace chr = std::chrono;
-ConsumeJournald::ConsumeJournald(const std::string_view name, const utils::Identifier &id, std::unique_ptr<libwrapper::LibWrapper>&& libwrapper)
- : core::ProcessorImpl{name, id}, libwrapper_{std::move(libwrapper)} {
- logger_ = core::logging::LoggerFactory<ConsumeJournald>::getLogger(uuid_);
-}
+ConsumeJournald::ConsumeJournald(core::ProcessorMetadata info, std::unique_ptr<libwrapper::LibWrapper>&& libwrapper)
+ : core::ProcessorImpl{info}, libwrapper_{std::move(libwrapper)} {}
void ConsumeJournald::initialize() {
setSupportedProperties(Properties);
@@ -95,13 +93,13 @@ void ConsumeJournald::onSchedule(core::ProcessContext& context, core::ProcessSes
running_ = true;
}
-void ConsumeJournald::onTrigger(core::ProcessContext&, core::ProcessSession& session) {
+void ConsumeJournald::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
gsl_Expects(state_manager_);
if (!running_.load(std::memory_order_acquire)) return;
auto cursor_and_messages = getCursorAndMessageBatch().get();
auto messages = std::move(cursor_and_messages.second);
if (messages.empty()) {
- yield();
+ context.yield();
return;
}
diff --git a/extensions/systemd/ConsumeJournald.h b/extensions/systemd/ConsumeJournald.h
index 16a3f2e10..07cd49dc4 100644
--- a/extensions/systemd/ConsumeJournald.h
+++ b/extensions/systemd/ConsumeJournald.h
@@ -35,7 +35,6 @@
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
#include "core/RelationshipDefinition.h"
-#include "core/logging/LoggerFactory.h"
#include "libwrapper/LibWrapper.h"
#include "utils/Deleters.h"
#include "utils/gsl.h"
@@ -114,7 +113,7 @@ class ConsumeJournald final : public core::ProcessorImpl {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit ConsumeJournald(std::string_view name, const utils::Identifier& id = {}, std::unique_ptr<libwrapper::LibWrapper>&& = libwrapper::createLibWrapper());
+ explicit ConsumeJournald(core::ProcessorMetadata info, std::unique_ptr<libwrapper::LibWrapper>&& = libwrapper::createLibWrapper());
ConsumeJournald(const ConsumeJournald&) = delete;
ConsumeJournald(ConsumeJournald&&) = delete;
ConsumeJournald& operator=(const ConsumeJournald&) = delete;
diff --git a/extensions/systemd/tests/ConsumeJournaldTest.cpp b/extensions/systemd/tests/ConsumeJournaldTest.cpp
index e6526b5fa..24c95f16f 100644
--- a/extensions/systemd/tests/ConsumeJournaldTest.cpp
+++ b/extensions/systemd/tests/ConsumeJournaldTest.cpp
@@ -160,11 +160,11 @@ TEST_CASE("ConsumeJournald", "[consumejournald]") {
{"systemd", "Starting Rule-based Manager for Device Events and Files...", 1},
}});
auto* const libwrapper_observer = libwrapper.get();
- const auto consume_journald = plan->addProcessor(std::make_unique<ConsumeJournald>("ConsumeJournald", utils::Identifier{},
+ const TypedProcessorWrapper<ConsumeJournald> consume_journald = plan->addProcessor(minifi::test::utils::make_custom_processor<ConsumeJournald>("ConsumeJournald", utils::Identifier{},
std::move(libwrapper)), "ConsumeJournald");
REQUIRE(consume_journald->setProperty(ConsumeJournald::TimestampFormat.name, "ISO8601"));
const auto get_cursor_position = [&consume_journald]() -> std::string {
- return ConsumeJournaldTestAccessor::get_state_manager_(dynamic_cast<ConsumeJournald&>(*consume_journald))->get()->at("cursor");
+ return ConsumeJournaldTestAccessor::get_state_manager_(consume_journald.get())->get()->at("cursor");
};
SECTION("defaults") {
(nifi-minifi-cpp) 13/37: Build fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit d635fda2514c911a3be8230e9ab5d236d66bdfca
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Apr 28 13:16:18 2025 +0200
Build fix
---
extensions/systemd/tests/ConsumeJournaldTest.cpp | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/extensions/systemd/tests/ConsumeJournaldTest.cpp b/extensions/systemd/tests/ConsumeJournaldTest.cpp
index 24c95f16f..3bed22c00 100644
--- a/extensions/systemd/tests/ConsumeJournaldTest.cpp
+++ b/extensions/systemd/tests/ConsumeJournaldTest.cpp
@@ -160,7 +160,8 @@ TEST_CASE("ConsumeJournald", "[consumejournald]") {
{"systemd", "Starting Rule-based Manager for Device Events and Files...", 1},
}});
auto* const libwrapper_observer = libwrapper.get();
- const TypedProcessorWrapper<ConsumeJournald> consume_journald = plan->addProcessor(minifi::test::utils::make_custom_processor<ConsumeJournald>("ConsumeJournald", utils::Identifier{},
+ const TypedProcessorWrapper<ConsumeJournald> consume_journald = plan->addProcessor(minifi::test::utils::make_custom_processor<ConsumeJournald>(
+ minifi::core::ProcessorMetadata{utils::Identifier{}, "ConsumeJournald", core::logging::LoggerFactory<ConsumeJournald>::getLogger()},
std::move(libwrapper)), "ConsumeJournald");
REQUIRE(consume_journald->setProperty(ConsumeJournald::TimestampFormat.name, "ISO8601"));
const auto get_cursor_position = [&consume_journald]() -> std::string {
(nifi-minifi-cpp) 32/37: Review changes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 4438275708e399067507814e442c2a8041c2a5ae
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Jul 17 15:17:21 2025 +0200
Review changes
---
core-framework/include/core/ProcessorImpl.h | 7 ++-----
extensions/python/PythonObjectFactory.h | 2 +-
libminifi/src/core/flow/StructuredConnectionParser.cpp | 2 +-
libminifi/test/libtest/unit/TestBase.h | 1 +
libminifi/test/libtest/unit/TestUtils.h | 5 +++--
5 files changed, 8 insertions(+), 9 deletions(-)
diff --git a/core-framework/include/core/ProcessorImpl.h b/core-framework/include/core/ProcessorImpl.h
index 987bb7586..fc6fc8a80 100644
--- a/core-framework/include/core/ProcessorImpl.h
+++ b/core-framework/include/core/ProcessorImpl.h
@@ -17,18 +17,15 @@
#pragma once
#include <algorithm>
+#include <array>
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <functional>
#include <memory>
#include <mutex>
+#include <span>
#include <string>
-#include <string_view>
-#include <unordered_set>
-#include <unordered_map>
-#include <utility>
-#include <vector>
#include "core/ConfigurableComponentImpl.h"
#include "core/Connectable.h"
diff --git a/extensions/python/PythonObjectFactory.h b/extensions/python/PythonObjectFactory.h
index 577f53f9a..8073dc6af 100644
--- a/extensions/python/PythonObjectFactory.h
+++ b/extensions/python/PythonObjectFactory.h
@@ -27,7 +27,7 @@
#include "core/ClassLoader.h"
#include "ExecutePythonProcessor.h"
#include "utils/StringUtils.h"
-#include "core/ProcessorFactory.h"
+#include "core/ProcessorFactoryImpl.h"
enum class PythonProcessorType {
MINIFI_TYPE,
diff --git a/libminifi/src/core/flow/StructuredConnectionParser.cpp b/libminifi/src/core/flow/StructuredConnectionParser.cpp
index 728bc95f6..4867e54df 100644
--- a/libminifi/src/core/flow/StructuredConnectionParser.cpp
+++ b/libminifi/src/core/flow/StructuredConnectionParser.cpp
@@ -41,7 +41,7 @@ void StructuredConnectionParser::addFunnelRelationshipToConnection(minifi::Conne
return;
}
- if (dynamic_cast<minifi::Funnel*>(&processor->getImpl())) {
+ if (typeid(minifi::Funnel) == typeid(processor->getImpl())) {
addNewRelationshipToConnection(minifi::Funnel::Success.name, connection);
}
}
diff --git a/libminifi/test/libtest/unit/TestBase.h b/libminifi/test/libtest/unit/TestBase.h
index 8ecef5ab6..39b2856af 100644
--- a/libminifi/test/libtest/unit/TestBase.h
+++ b/libminifi/test/libtest/unit/TestBase.h
@@ -205,6 +205,7 @@ template<typename T>
class TypedProcessorWrapper {
public:
TypedProcessorWrapper() = default;
+ // it's a wrapper over the same object, with no value change
TypedProcessorWrapper(core::Processor* proc): proc_(proc) { // NOLINT(runtime/explicit)
if (proc_) {
proc_->getImpl<T>();
diff --git a/libminifi/test/libtest/unit/TestUtils.h b/libminifi/test/libtest/unit/TestUtils.h
index 83e9bce23..13d7b7d24 100644
--- a/libminifi/test/libtest/unit/TestUtils.h
+++ b/libminifi/test/libtest/unit/TestUtils.h
@@ -213,10 +213,11 @@ inline std::error_code hide_file(const std::filesystem::path& file_name) {
#endif /* WIN32 */
template<typename T>
-concept NetworkingProcessor = requires(T x) {
+concept NetworkingProcessor = std::derived_from<T, minifi::core::ProcessorApi>
+ && requires(T x) {
{T::Port} -> std::convertible_to<core::PropertyReference>;
{x.getPort()} -> std::convertible_to<uint16_t>;
-}; // NOLINT(readability/braces)
+ }; // NOLINT(readability/braces)
template<NetworkingProcessor T>
uint16_t scheduleProcessorOnRandomPort(const std::shared_ptr<TestPlan>& test_plan, const TypedProcessorWrapper<T>& processor) {
(nifi-minifi-cpp) 03/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 7be07f99032e68468c2916504f12be3a1f7e1751
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Mar 6 12:22:12 2025 +0100
Rebase fix
---
extensions/python/PythonCreator.h | 2 +-
libminifi/src/Funnel.cpp | 5 +++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/extensions/python/PythonCreator.h b/extensions/python/PythonCreator.h
index b326d34dc..647f3255a 100644
--- a/extensions/python/PythonCreator.h
+++ b/extensions/python/PythonCreator.h
@@ -124,7 +124,7 @@ class PythonCreator : public minifi::core::CoreComponentImpl {
factory = std::make_unique<PythonObjectFactory>(path.string(), script_name.string(),
PythonProcessorType::MINIFI_TYPE, std::vector<std::filesystem::path>{python_lib_path, std::filesystem::path{pathListings.value()}}, qualified_module_name);
}
- if (!registered_classes_.emplace(class_name, factory.get()).second) {
+ if (registered_classes_.emplace(class_name, factory.get()).second) {
core::getClassLoader().registerClass(class_name, std::move(factory));
} else {
logger_->log_warn("Ignoring duplicate python processor at '{}' with class '{}'", path.string(), class_name);
diff --git a/libminifi/src/Funnel.cpp b/libminifi/src/Funnel.cpp
index fee7ccf78..a3ef769f3 100644
--- a/libminifi/src/Funnel.cpp
+++ b/libminifi/src/Funnel.cpp
@@ -16,11 +16,12 @@
*/
#include "Funnel.h"
+#include "minifi-cpp/core/ProcessContext.h"
namespace org::apache::nifi::minifi {
-void Funnel::onSchedule(core::ProcessContext&, core::ProcessSessionFactory&) {
- addAutoTerminatedRelationship(Success);
+void Funnel::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) {
+ context.addAutoTerminatedRelationship(Success);
}
} // namespace org::apache::nifi::minifi
(nifi-minifi-cpp) 05/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit ff298550549ad3b864c5bd9efd97debc9e41dc6f
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Mar 20 08:39:42 2025 +0100
Rebase fix
---
.../include/utils/ProcessorConfigUtils.h | 24 ++---
extension-utils/src/utils/ProcessorConfigUtils.cpp | 108 ---------------------
.../expression-language/ProcessContextExpr.cpp | 6 +-
.../tests/unit/FlowJsonTests.cpp | 2 +-
.../tests/unit/YamlConfigurationTests.cpp | 2 +-
libminifi/src/core/ProcessContext.cpp | 4 +-
libminifi/test/libtest/unit/MockClasses.h | 2 +-
.../include/minifi-cpp/core/ProcessContext.h | 4 +-
8 files changed, 22 insertions(+), 130 deletions(-)
diff --git a/extension-utils/include/utils/ProcessorConfigUtils.h b/extension-utils/include/utils/ProcessorConfigUtils.h
index 29de57eec..5f0ca845f 100644
--- a/extension-utils/include/utils/ProcessorConfigUtils.h
+++ b/extension-utils/include/utils/ProcessorConfigUtils.h
@@ -32,27 +32,27 @@
namespace org::apache::nifi::minifi::utils {
inline std::string parseProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file = nullptr) {
- return ctx.getProperty(property.name, flow_file) | orThrow(fmt::format("Expected valid value from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return ctx.getProperty(property.name, flow_file) | orThrow(fmt::format("Expected valid value from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
inline bool parseBoolProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file = nullptr) {
- return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseBool) | orThrow(fmt::format("Expected parsable bool from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseBool) | orThrow(fmt::format("Expected parsable bool from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
inline uint64_t parseU64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file = nullptr) {
- return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseIntegral<uint64_t>) | orThrow(fmt::format("Expected parsable uint64_t from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseIntegral<uint64_t>) | orThrow(fmt::format("Expected parsable uint64_t from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
inline int64_t parseI64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file = nullptr) {
- return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseIntegral<int64_t>) | orThrow(fmt::format("Expected parsable int64_t from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseIntegral<int64_t>) | orThrow(fmt::format("Expected parsable int64_t from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
inline std::chrono::milliseconds parseDurationProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file = nullptr) {
- return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseDuration<std::chrono::milliseconds>) | orThrow(fmt::format("Expected parsable duration from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseDuration<std::chrono::milliseconds>) | orThrow(fmt::format("Expected parsable duration from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
inline uint64_t parseDataSizeProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file = nullptr) {
- return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseDataSize) | orThrow(fmt::format("Expected parsable data size from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return ctx.getProperty(property.name, flow_file) | andThen(parsing::parseDataSize) | orThrow(fmt::format("Expected parsable data size from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
inline std::optional<std::string> parseOptionalProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file = nullptr) {
@@ -61,7 +61,7 @@ inline std::optional<std::string> parseOptionalProperty(const core::ProcessConte
inline std::optional<bool> parseOptionalBoolProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file = nullptr) {
if (const auto property_str = ctx.getProperty(property.name, flow_file)) {
- return parsing::parseBool(*property_str) | utils::orThrow(fmt::format("Expected parsable bool from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return parsing::parseBool(*property_str) | utils::orThrow(fmt::format("Expected parsable bool from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
return std::nullopt;
}
@@ -71,7 +71,7 @@ inline std::optional<uint64_t> parseOptionalU64Property(const core::ProcessConte
if (property_str->empty()) {
return std::nullopt;
}
- return parsing::parseIntegral<uint64_t>(*property_str) | utils::orThrow(fmt::format("Expected parsable uint64_t from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return parsing::parseIntegral<uint64_t>(*property_str) | utils::orThrow(fmt::format("Expected parsable uint64_t from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
return std::nullopt;
@@ -82,7 +82,7 @@ inline std::optional<int64_t> parseOptionalI64Property(const core::ProcessContex
if (property_str->empty()) {
return std::nullopt;
}
- return parsing::parseIntegral<int64_t>(*property_str) | utils::orThrow(fmt::format("Expected parsable int64_t from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return parsing::parseIntegral<int64_t>(*property_str) | utils::orThrow(fmt::format("Expected parsable int64_t from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
return std::nullopt;
@@ -93,7 +93,7 @@ inline std::optional<std::chrono::milliseconds> parseOptionalDurationProperty(co
if (property_str->empty()) {
return std::nullopt;
}
- return parsing::parseDuration(*property_str) | utils::orThrow(fmt::format("Expected parsable duration from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return parsing::parseDuration(*property_str) | utils::orThrow(fmt::format("Expected parsable duration from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
return std::nullopt;
@@ -104,7 +104,7 @@ inline std::optional<uint64_t> parseOptionalDataSizeProperty(const core::Process
if (property_str->empty()) {
return std::nullopt;
}
- return parsing::parseDataSize(*property_str) | utils::orThrow(fmt::format("Expected parsable data size from \"{}::{}\"", ctx.getProcessor().getName(), property.name));
+ return parsing::parseDataSize(*property_str) | utils::orThrow(fmt::format("Expected parsable data size from \"{}::{}\"", ctx.getProcessorInfo().getName(), property.name));
}
return std::nullopt;
@@ -115,7 +115,7 @@ inline std::optional<float> parseOptionalFloatProperty(const core::ProcessContex
if (property_str->empty()) {
return std::nullopt;
}
- return parsing::parseFloat(*property_str) | utils::orThrow(fmt::format("Expected parsable float from {}::{}", ctx.getProcessor().getName(), property.name));
+ return parsing::parseFloat(*property_str) | utils::orThrow(fmt::format("Expected parsable float from {}::{}", ctx.getProcessorInfo().getName(), property.name));
}
return std::nullopt;
}
diff --git a/extension-utils/src/utils/ProcessorConfigUtils.cpp b/extension-utils/src/utils/ProcessorConfigUtils.cpp
deleted file mode 100644
index 2ba8c73cb..000000000
--- a/extension-utils/src/utils/ProcessorConfigUtils.cpp
+++ /dev/null
@@ -1,108 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-#include "utils/ProcessorConfigUtils.h"
-
-#include <string>
-
-#include "utils/expected.h"
-
-namespace org::apache::nifi::minifi::utils {
-
-std::string parseProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- return ctx.getProperty(property, flow_file) | utils::expect(fmt::format("Expected valid value from {}::{}", ctx.getProcessorInfo().getName(), property.name));
-}
-
-std::optional<std::string> parseOptionalProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- if (const auto property_str = ctx.getProperty(property, flow_file)) {
- return *property_str;
- }
- return std::nullopt;
-}
-
-std::optional<bool> parseOptionalBoolProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- if (const auto property_str = ctx.getProperty(property, flow_file)) {
- return parsing::parseBool(*property_str) | utils::expect(fmt::format("Expected parsable bool from {}::{}", ctx.getProcessorInfo().getName(), property.name));
- }
- return std::nullopt;
-}
-
-bool parseBoolProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseBool) | utils::expect(fmt::format("Expected parsable bool from {}::{}", ctx.getProcessorInfo().getName(), property.name));
-}
-
-std::optional<uint64_t> parseOptionalU64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- if (const auto property_str = ctx.getProperty(property, flow_file)) {
- if (property_str->empty()) {
- return std::nullopt;
- }
- return parsing::parseIntegral<uint64_t>(*property_str) | utils::expect(fmt::format("Expected parsable uint64_t from {}::{}", ctx.getProcessorInfo().getName(), property.name));
- }
-
- return std::nullopt;
-}
-
-uint64_t parseU64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseIntegral<uint64_t>) | utils::expect(fmt::format("Expected parsable uint64_t from {}::{}", ctx.getProcessorInfo().getName(), property.name));
-}
-
-std::optional<int64_t> parseOptionalI64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- if (const auto property_str = ctx.getProperty(property, flow_file)) {
- if (property_str->empty()) {
- return std::nullopt;
- }
- return parsing::parseIntegral<int64_t>(*property_str) | utils::expect(fmt::format("Expected parsable int64_t from {}::{}", ctx.getProcessorInfo().getName(), property.name));
- }
-
- return std::nullopt;
-}
-
-int64_t parseI64Property(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseIntegral<int64_t>) | utils::expect(fmt::format("Expected parsable int64_t from {}::{}", ctx.getProcessorInfo().getName(), property.name));
-}
-
-std::optional<std::chrono::milliseconds> parseOptionalMsProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- if (const auto property_str = ctx.getProperty(property, flow_file)) {
- if (property_str->empty()) {
- return std::nullopt;
- }
- return parsing::parseDuration(*property_str) | utils::expect(fmt::format("Expected parsable duration from {}::{}", ctx.getProcessorInfo().getName(), property.name));
- }
-
- return std::nullopt;
-}
-
-std::chrono::milliseconds parseMsProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseDuration<std::chrono::milliseconds>) | utils::expect(fmt::format("Expected parsable duration from {}::{}", ctx.getProcessorInfo().getName(), property.name));
-}
-
-std::optional<uint64_t> parseOptionalDataSizeProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- if (const auto property_str = ctx.getProperty(property, flow_file)) {
- if (property_str->empty()) {
- return std::nullopt;
- }
- return parsing::parseDataSize(*property_str) | utils::expect(fmt::format("Expected parsable data size from {}::{}", ctx.getProcessorInfo().getName(), property.name));
- }
-
- return std::nullopt;
-}
-
-uint64_t parseDataSizeProperty(const core::ProcessContext& ctx, const core::PropertyReference& property, const core::FlowFile* flow_file) {
- return ctx.getProperty(property, flow_file) | utils::andThen(parsing::parseDataSize) | utils::expect(fmt::format("Expected parsable data size from {}::{}", ctx.getProcessorInfo().getName(), property.name));
-}
-
-
-} // namespace org::apache::nifi::minifi::utils
diff --git a/extensions/expression-language/ProcessContextExpr.cpp b/extensions/expression-language/ProcessContextExpr.cpp
index 04deec7f6..fb917045e 100644
--- a/extensions/expression-language/ProcessContextExpr.cpp
+++ b/extensions/expression-language/ProcessContextExpr.cpp
@@ -26,12 +26,12 @@
namespace org::apache::nifi::minifi::core {
nonstd::expected<std::string, std::error_code> ProcessContextExpr::getProperty(const std::string_view name, const FlowFile* flow_file) const {
- const auto property = getProcessorInfo().getPropertyReference(name);
+ const auto property = getProcessorInfo().getSupportedProperty(name);
if (!property) {
return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty);
}
- if (!property->supports_expression_language) {
+ if (!property->supportsExpressionLanguage()) {
return ProcessContextImpl::getProperty(name, flow_file);
}
if (!cached_expressions_.contains(name)) {
@@ -41,7 +41,7 @@ nonstd::expected<std::string, std::error_code> ProcessContextExpr::getProperty(c
}
expression::Parameters p(this, flow_file);
auto result = cached_expressions_[std::string{name}](p).asString();
- if (!property->validator->validate(result)) {
+ if (!property->getValidator().validate(result)) {
return nonstd::make_unexpected(PropertyErrorCode::ValidationFailed);
}
return result;
diff --git a/extensions/standard-processors/tests/unit/FlowJsonTests.cpp b/extensions/standard-processors/tests/unit/FlowJsonTests.cpp
index c36dc6664..05cccb361 100644
--- a/extensions/standard-processors/tests/unit/FlowJsonTests.cpp
+++ b/extensions/standard-processors/tests/unit/FlowJsonTests.cpp
@@ -1532,7 +1532,7 @@ TEST_CASE("Parameter providers can be used for parameter values") {
std::unique_ptr<core::ProcessGroup> flow = config.getRootFromPayload(CONFIG_JSON);
REQUIRE(flow);
- auto* proc = dynamic_cast<core::ProcessorImpl*>(flow->findProcessorByName("MyProcessor"));
+ auto* proc = flow->findProcessorByName("MyProcessor");
REQUIRE(proc);
auto values = proc->getAllDynamicPropertyValues("My Dynamic Property Sequence");
CHECK((*values)[0] == "value2");
diff --git a/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp b/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp
index 21137ad17..53c6118c3 100644
--- a/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp
+++ b/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp
@@ -2150,7 +2150,7 @@ Parameter Context Name: dummycontext
std::unique_ptr<core::ProcessGroup> flow = yaml_config.getRootFromPayload(TEST_CONFIG_YAML);
REQUIRE(flow);
- auto* proc = dynamic_cast<core::ProcessorImpl*>(flow->findProcessorByName("DummyProcessor"));
+ auto* proc = flow->findProcessorByName("DummyProcessor");
REQUIRE(proc);
auto values = proc->getAllDynamicPropertyValues("My Dynamic Property Sequence");
CHECK((*values)[0] == "value2");
diff --git a/libminifi/src/core/ProcessContext.cpp b/libminifi/src/core/ProcessContext.cpp
index 39e993108..dd9d55099 100644
--- a/libminifi/src/core/ProcessContext.cpp
+++ b/libminifi/src/core/ProcessContext.cpp
@@ -29,8 +29,8 @@ class StandardProcessorInfo : public ProcessorInfo {
std::string getName() const override {return proc_.getName();}
utils::Identifier getUUID() const override {return proc_.getUUID();}
std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const override {return proc_.getFlowIdentifier();}
- std::map<std::string, core::Property> getSupportedProperties() const override {return proc_.getSupportedProperties();}
- nonstd::expected<PropertyReference, std::error_code> getPropertyReference(std::string_view name) const override {return proc_.getPropertyReference(name);}
+ std::map<std::string, core::Property, std::less<>> getSupportedProperties() const override {return proc_.getSupportedProperties();}
+ nonstd::expected<Property, std::error_code> getSupportedProperty(std::string_view name) const override {return proc_.getSupportedProperty(name);}
private:
Processor& proc_;
diff --git a/libminifi/test/libtest/unit/MockClasses.h b/libminifi/test/libtest/unit/MockClasses.h
index 77c63e5ba..bb55e7a30 100644
--- a/libminifi/test/libtest/unit/MockClasses.h
+++ b/libminifi/test/libtest/unit/MockClasses.h
@@ -125,7 +125,7 @@ class MockProcessor : public minifi::core::ProcessorImpl {
// and verify that we can execute it.
}
- bool in_sub_process_group = context.getProperty("InSubProcessGroup") | minifi::utils::andThen(minifi::parsing::parseBool) | minifi::utils::expect("");
+ bool in_sub_process_group = context.getProperty("InSubProcessGroup") | minifi::utils::andThen(minifi::parsing::parseBool) | minifi::utils::orThrow("");
auto sub_service = context.getControllerService("SubMockController", getUUID());
if (in_sub_process_group) {
REQUIRE(nullptr != sub_service);
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContext.h b/minifi-api/include/minifi-cpp/core/ProcessContext.h
index 629d06c28..f4f963e1d 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContext.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContext.h
@@ -42,8 +42,8 @@ class ProcessorInfo {
virtual std::string getName() const = 0;
virtual utils::Identifier getUUID() const = 0;
virtual std::shared_ptr<state::FlowIdentifier> getFlowIdentifier() const = 0;
- virtual std::map<std::string, core::Property> getSupportedProperties() const = 0;
- virtual nonstd::expected<PropertyReference, std::error_code> getPropertyReference(std::string_view name) const = 0;
+ virtual std::map<std::string, core::Property, std::less<>> getSupportedProperties() const = 0;
+ virtual nonstd::expected<Property, std::error_code> getSupportedProperty(std::string_view name) const = 0;
virtual ~ProcessorInfo() = default;
};
(nifi-minifi-cpp) 17/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 6a55322642230ae44b792d0d5b390d7a6f09ed81
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Jun 19 15:57:53 2025 +0200
Rebase fix
---
extensions/aws/processors/AwsProcessor.cpp | 5 ---
extensions/aws/processors/AwsProcessor.h | 6 +--
extensions/aws/processors/PutKinesisStream.h | 8 +---
extensions/aws/processors/S3Processor.cpp | 2 +-
extensions/aws/processors/S3Processor.h | 1 -
extensions/aws/tests/PutKinesisStreamTests.cpp | 39 ++++++++----------
extensions/kafka/ConsumeKafka.cpp | 2 +-
extensions/python/ExecutePythonProcessor.h | 5 ++-
extensions/python/PythonCreator.h | 4 ++
libminifi/include/core/Processor.h | 10 +++++
libminifi/src/core/ProcessContext.cpp | 4 ++
libminifi/src/core/Processor.cpp | 28 +++++++++++++
.../integration/C2ControllerEnableFailureTest.cpp | 9 +----
libminifi/test/integration/C2MetricsTest.cpp | 46 ++++++++++++----------
libminifi/test/unit/BulletinStoreTests.cpp | 3 +-
.../include/minifi-cpp/core/ProcessContext.h | 2 +
minifi-api/include/minifi-cpp/core/ProcessorApi.h | 1 +
utils/include/core/ProcessContext.h | 1 +
utils/include/core/ProcessorImpl.h | 2 +
19 files changed, 106 insertions(+), 72 deletions(-)
diff --git a/extensions/aws/processors/AwsProcessor.cpp b/extensions/aws/processors/AwsProcessor.cpp
index 4ff850ee0..cd442abb7 100644
--- a/extensions/aws/processors/AwsProcessor.cpp
+++ b/extensions/aws/processors/AwsProcessor.cpp
@@ -32,11 +32,6 @@
namespace org::apache::nifi::minifi::aws::processors {
-AwsProcessor::AwsProcessor(std::string_view name, const minifi::utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger)
- : core::ProcessorImpl(name, uuid),
- logger_(std::move(logger)) {
-}
-
std::optional<Aws::Auth::AWSCredentials> AwsProcessor::getAWSCredentialsFromControllerService(core::ProcessContext& context) const {
if (const auto aws_credentials_service = minifi::utils::parseOptionalControllerService<controllers::AWSCredentialsService>(context, AWSCredentialsProviderService, getUUID())) {
return (*aws_credentials_service)->getAWSCredentials();
diff --git a/extensions/aws/processors/AwsProcessor.h b/extensions/aws/processors/AwsProcessor.h
index 9f0055e2e..542d0c75b 100644
--- a/extensions/aws/processors/AwsProcessor.h
+++ b/extensions/aws/processors/AwsProcessor.h
@@ -32,7 +32,7 @@
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
namespace org::apache::nifi::minifi::aws::processors {
@@ -169,8 +169,7 @@ class AwsProcessor : public core::ProcessorImpl {
UseDefaultCredentials
});
-
- explicit AwsProcessor(std::string_view name, const minifi::utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger);
+ using ProcessorImpl::ProcessorImpl;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
@@ -180,7 +179,6 @@ class AwsProcessor : public core::ProcessorImpl {
aws::ProxyOptions getProxy(core::ProcessContext& context, const core::FlowFile* const flow_file);
std::optional<CommonProperties> getCommonELSupportedProperties(core::ProcessContext& context, const core::FlowFile* flow_file);
- std::shared_ptr<core::logging::Logger> logger_;
std::optional<Aws::Client::ClientConfiguration> client_config_;
};
diff --git a/extensions/aws/processors/PutKinesisStream.h b/extensions/aws/processors/PutKinesisStream.h
index c30483a20..73d308dcf 100644
--- a/extensions/aws/processors/PutKinesisStream.h
+++ b/extensions/aws/processors/PutKinesisStream.h
@@ -87,13 +87,7 @@ class PutKinesisStream : public AwsProcessor {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PutKinesisStream(const std::string& name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : AwsProcessor(name, uuid, core::logging::LoggerFactory<PutKinesisStream>::getLogger(uuid)) {
- }
- PutKinesisStream(const PutKinesisStream&) = delete;
- PutKinesisStream(PutKinesisStream&&) = delete;
- PutKinesisStream& operator=(const PutKinesisStream&) = delete;
- PutKinesisStream& operator=(PutKinesisStream&&) = delete;
+ using AwsProcessor::AwsProcessor;
~PutKinesisStream() override = default;
void initialize() override;
diff --git a/extensions/aws/processors/S3Processor.cpp b/extensions/aws/processors/S3Processor.cpp
index e45c3fa1c..6bb6402f4 100644
--- a/extensions/aws/processors/S3Processor.cpp
+++ b/extensions/aws/processors/S3Processor.cpp
@@ -33,7 +33,7 @@
namespace org::apache::nifi::minifi::aws::processors {
S3Processor::S3Processor(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : core::ProcessorImpl(info),
+ : AwsProcessor(info),
s3_wrapper_(std::move(s3_request_sender)) {
}
diff --git a/extensions/aws/processors/S3Processor.h b/extensions/aws/processors/S3Processor.h
index d3d0a485b..cdcee4c4c 100644
--- a/extensions/aws/processors/S3Processor.h
+++ b/extensions/aws/processors/S3Processor.h
@@ -30,7 +30,6 @@
#include "AwsProcessor.h"
#include "S3Wrapper.h"
#include "aws/core/auth/AWSCredentialsProvider.h"
-#include "core/Processor.h"
#include "core/Property.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/aws/tests/PutKinesisStreamTests.cpp b/extensions/aws/tests/PutKinesisStreamTests.cpp
index bbfda2b9c..b6532b5b3 100644
--- a/extensions/aws/tests/PutKinesisStreamTests.cpp
+++ b/extensions/aws/tests/PutKinesisStreamTests.cpp
@@ -24,6 +24,7 @@
#include "unit/Catch.h"
#include "unit/SingleProcessorTestController.h"
#include "unit/TestBase.h"
+#include "unit/ProcessorUtils.h"
namespace org::apache::nifi::minifi::aws::processors::test {
@@ -86,14 +87,7 @@ class PutKinesisStreamMocked final : public aws::processors::PutKinesisStream {
public:
static constexpr const char* Description = "PutKinesisStreamMocked";
- explicit PutKinesisStreamMocked(const std::string& name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : PutKinesisStream(name, uuid) {
- }
-
- PutKinesisStreamMocked(const PutKinesisStreamMocked&) = delete;
- PutKinesisStreamMocked(PutKinesisStreamMocked&&) = delete;
- PutKinesisStreamMocked& operator=(const PutKinesisStreamMocked&) = delete;
- PutKinesisStreamMocked& operator=(PutKinesisStreamMocked&&) = delete;
+ using PutKinesisStream::PutKinesisStream;
~PutKinesisStreamMocked() override = default;
@@ -110,10 +104,10 @@ class PutKinesisStreamMocked final : public aws::processors::PutKinesisStream {
REGISTER_RESOURCE(PutKinesisStreamMocked, Processor);
TEST_CASE("PutKinesisStream record size mismatch path") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<PutKinesisStreamMocked>("PutKinesisStream"));
- auto put_kinesis_stream = dynamic_cast<PutKinesisStreamMocked*>(controller.getProcessor());
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_processor<PutKinesisStreamMocked>("PutKinesisStream"));
+ auto put_kinesis_stream = controller.getProcessor<PutKinesisStreamMocked>();
REQUIRE(put_kinesis_stream);
- put_kinesis_stream->behaviour_ = MockKinesisClient::KinesisBehaviour::RecordSizeMismatch;
+ put_kinesis_stream.get().behaviour_ = MockKinesisClient::KinesisBehaviour::RecordSizeMismatch;
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::AccessKey, "access_key");
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::SecretKey, "secret_key");
@@ -134,10 +128,10 @@ TEST_CASE("PutKinesisStream record size mismatch path") {
}
TEST_CASE("PutKinesisStream record size failure path") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<PutKinesisStreamMocked>("PutKinesisStream"));
- auto put_kinesis_stream = dynamic_cast<PutKinesisStreamMocked*>(controller.getProcessor());
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_processor<PutKinesisStreamMocked>("PutKinesisStream"));
+ auto put_kinesis_stream = controller.getProcessor<PutKinesisStreamMocked>();
REQUIRE(put_kinesis_stream);
- put_kinesis_stream->behaviour_ = MockKinesisClient::KinesisBehaviour::Failure;
+ put_kinesis_stream.get().behaviour_ = MockKinesisClient::KinesisBehaviour::Failure;
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::AccessKey, "access_key");
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::SecretKey, "secret_key");
@@ -160,10 +154,9 @@ TEST_CASE("PutKinesisStream record size failure path") {
}
TEST_CASE("PutKinesisStream partial failure path") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<PutKinesisStreamMocked>("PutKinesisStream"));
- auto put_kinesis_stream = dynamic_cast<PutKinesisStreamMocked*>(controller.getProcessor());
- REQUIRE(put_kinesis_stream);
- put_kinesis_stream->behaviour_ = MockKinesisClient::KinesisBehaviour::OddsFail;
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_processor<PutKinesisStreamMocked>("PutKinesisStream"));
+ auto put_kinesis_stream = controller.getProcessor<PutKinesisStreamMocked>();
+ put_kinesis_stream.get().behaviour_ = MockKinesisClient::KinesisBehaviour::OddsFail;
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::AccessKey, "access_key");
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::SecretKey, "secret_key");
@@ -187,7 +180,7 @@ TEST_CASE("PutKinesisStream partial failure path") {
TEST_CASE("PutKinesisStream simple happy path") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<PutKinesisStreamMocked>("PutKinesisStream"));
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_processor<PutKinesisStreamMocked>("PutKinesisStream"));
auto put_kinesis_stream = controller.getProcessor();
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::AccessKey, "access_key");
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::SecretKey, "secret_key");
@@ -210,7 +203,7 @@ TEST_CASE("PutKinesisStream simple happy path") {
}
TEST_CASE("PutKinesisStream smaller batch size than available ffs") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<PutKinesisStreamMocked>("PutKinesisStream"));
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_processor<PutKinesisStreamMocked>("PutKinesisStream"));
auto put_kinesis_stream = controller.getProcessor();
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::AccessKey, "access_key");
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::SecretKey, "secret_key");
@@ -237,7 +230,7 @@ TEST_CASE("PutKinesisStream smaller batch size than available ffs") {
}
TEST_CASE("PutKinesisStream max batch data size fills up") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<PutKinesisStreamMocked>("PutKinesisStream"));
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_processor<PutKinesisStreamMocked>("PutKinesisStream"));
auto put_kinesis_stream = controller.getProcessor();
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::AccessKey, "access_key");
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::SecretKey, "secret_key");
@@ -268,7 +261,7 @@ TEST_CASE("PutKinesisStream max batch data size fills up") {
}
TEST_CASE("PutKinesisStream max batch data size to different streams") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<PutKinesisStreamMocked>("PutKinesisStream"));
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_processor<PutKinesisStreamMocked>("PutKinesisStream"));
auto put_kinesis_stream = controller.getProcessor();
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::AccessKey, "access_key");
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::SecretKey, "secret_key");
@@ -296,7 +289,7 @@ TEST_CASE("PutKinesisStream max batch data size to different streams") {
}
TEST_CASE("PutKinesisStream with too large message") {
- minifi::test::SingleProcessorTestController controller(std::make_unique<PutKinesisStreamMocked>("PutKinesisStream"));
+ minifi::test::SingleProcessorTestController controller(minifi::test::utils::make_processor<PutKinesisStreamMocked>("PutKinesisStream"));
auto put_kinesis_stream = controller.getProcessor();
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::AccessKey, "access_key");
controller.plan->setProperty(put_kinesis_stream, PutKinesisStream::SecretKey, "secret_key");
diff --git a/extensions/kafka/ConsumeKafka.cpp b/extensions/kafka/ConsumeKafka.cpp
index 76f317823..c8ef2f262 100644
--- a/extensions/kafka/ConsumeKafka.cpp
+++ b/extensions/kafka/ConsumeKafka.cpp
@@ -76,7 +76,7 @@ void ConsumeKafka::onSchedule(core::ProcessContext& context, core::ProcessSessio
configureNewConnection(context);
if (commit_policy_ == consume_kafka::CommitPolicyEnum::CommitFromIncomingFlowFiles) {
setTriggerWhenEmpty(true);
- } else if (hasIncomingConnections()) {
+ } else if (context.hasIncomingConnections()) {
logger_->log_error("Incoming connections are not allowed with {}", magic_enum::enum_name(commit_policy_));
throw Exception(PROCESS_SCHEDULE_EXCEPTION, fmt::format("Incoming connections are not allowed with {}", magic_enum::enum_name(commit_policy_)));
}
diff --git a/extensions/python/ExecutePythonProcessor.h b/extensions/python/ExecutePythonProcessor.h
index c54f1840f..1e44b8cc3 100644
--- a/extensions/python/ExecutePythonProcessor.h
+++ b/extensions/python/ExecutePythonProcessor.h
@@ -44,7 +44,9 @@ class ExecutePythonProcessor : public core::ProcessorImpl {
: ProcessorImpl(info),
processor_initialized_(false),
python_dynamic_(false),
- reload_on_script_change_(true) {}
+ reload_on_script_change_(true) {
+ python_logger_ = core::logging::LoggerFactory<ExecutePythonProcessor>::getAliasedLogger(getName(), info.uuid);
+ }
EXTENSIONAPI static constexpr const char* Description = "Executes a script given the flow file and a process session. "
"The script is responsible for handling the incoming flow file (transfer to SUCCESS or remove, e.g.) as well as "
@@ -157,6 +159,7 @@ class ExecutePythonProcessor : public core::ProcessorImpl {
bool reload_on_script_change_;
std::optional<std::chrono::file_clock::time_point> last_script_write_time_;
std::string script_file_path_;
+ std::shared_ptr<core::logging::Logger> python_logger_;
std::unique_ptr<PythonScriptEngine> python_script_engine_;
std::optional<std::string> python_class_name_;
std::vector<std::filesystem::path> python_paths_;
diff --git a/extensions/python/PythonCreator.h b/extensions/python/PythonCreator.h
index 362f737db..961928085 100644
--- a/extensions/python/PythonCreator.h
+++ b/extensions/python/PythonCreator.h
@@ -68,6 +68,10 @@ class DummyLogger : public core::logging::Logger {
return 0;
}
+ void setLogCallback(const std::function<void(core::logging::LOG_LEVEL level, const std::string&)>& /*callback*/) override {
+ // pass
+ }
+
~DummyLogger() override = default;
};
diff --git a/libminifi/include/core/Processor.h b/libminifi/include/core/Processor.h
index c9503dc18..9daec72c1 100644
--- a/libminifi/include/core/Processor.h
+++ b/libminifi/include/core/Processor.h
@@ -113,6 +113,13 @@ class Processor : public ConnectableImpl, public ConfigurableComponentImpl, publ
[[nodiscard]] bool supportsDynamicRelationships() const override;
state::response::SharedResponseNode getResponseNode() override;
gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const;
+ std::string getProcessGroupName() const;
+ void setProcessGroupName(const std::string &name);
+ std::string getProcessGroupPath() const;
+ void setProcessGroupPath(const std::string &path);
+ logging::LOG_LEVEL getLogBulletinLevel() const;
+ void setLogBulletinLevel(logging::LOG_LEVEL level);
+ void setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback);
void restore(const std::shared_ptr<FlowFile>& file) override;
static constexpr auto DynamicProperties = std::array<DynamicProperty, 0>{};
@@ -143,6 +150,7 @@ class Processor : public ConnectableImpl, public ConfigurableComponentImpl, publ
std::string cron_period_;
std::shared_ptr<logging::Logger> logger_;
+ logging::LOG_LEVEL log_bulletin_level_ = logging::LOG_LEVEL::warn;
private:
mutable std::mutex mutex_;
@@ -159,6 +167,8 @@ class Processor : public ConnectableImpl, public ConfigurableComponentImpl, publ
std::unordered_map<Connection*, std::unordered_set<Processor*>> reachable_processors_;
std::string process_group_uuid_;
+ std::string process_group_name_;
+ std::string process_group_path_;
protected:
std::unique_ptr<ProcessorApi> impl_;
diff --git a/libminifi/src/core/ProcessContext.cpp b/libminifi/src/core/ProcessContext.cpp
index 3e453d58c..985a23271 100644
--- a/libminifi/src/core/ProcessContext.cpp
+++ b/libminifi/src/core/ProcessContext.cpp
@@ -104,6 +104,10 @@ nonstd::expected<std::string, std::error_code> ProcessContextImpl::getRawPropert
return getProcessor().getProperty(name);
}
+nonstd::expected<std::string, std::error_code> ProcessContextImpl::getRawDynamicProperty(const std::string_view name) const {
+ return getProcessor().getDynamicProperty(name);
+}
+
nonstd::expected<void, std::error_code> ProcessContextImpl::setDynamicProperty(std::string name, std::string value) {
return getProcessor().setDynamicProperty(std::move(name), std::move(value));
}
diff --git a/libminifi/src/core/Processor.cpp b/libminifi/src/core/Processor.cpp
index 2ff4dc802..e0df1a394 100644
--- a/libminifi/src/core/Processor.cpp
+++ b/libminifi/src/core/Processor.cpp
@@ -469,6 +469,34 @@ void Processor::setProcessGroupUUIDStr(const std::string &uuid) {
process_group_uuid_ = uuid;
}
+std::string Processor::getProcessGroupName() const {
+ return process_group_name_;
+}
+
+void Processor::setProcessGroupName(const std::string &name) {
+ process_group_name_ = name;
+}
+
+std::string Processor::getProcessGroupPath() const {
+ return process_group_path_;
+}
+
+void Processor::setProcessGroupPath(const std::string &path) {
+ process_group_path_ = path;
+}
+
+logging::LOG_LEVEL Processor::getLogBulletinLevel() const {
+ return log_bulletin_level_;
+}
+
+void Processor::setLogBulletinLevel(logging::LOG_LEVEL level) {
+ log_bulletin_level_ = level;
+}
+
+void Processor::setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback) {
+ impl_->setLoggerCallback(callback);
+}
+
std::chrono::steady_clock::time_point Processor::getYieldExpirationTime() const {
return yield_expiration_;
}
diff --git a/libminifi/test/integration/C2ControllerEnableFailureTest.cpp b/libminifi/test/integration/C2ControllerEnableFailureTest.cpp
index 79267b20b..3cc01e1ff 100644
--- a/libminifi/test/integration/C2ControllerEnableFailureTest.cpp
+++ b/libminifi/test/integration/C2ControllerEnableFailureTest.cpp
@@ -72,11 +72,9 @@ class DummyController : public core::controller::ControllerServiceImpl {
REGISTER_RESOURCE(DummyController, ControllerService);
class DummmyControllerUserProcessor : public minifi::core::ProcessorImpl {
+ public:
using minifi::core::ProcessorImpl::ProcessorImpl;
- public:
- DummmyControllerUserProcessor(std::string_view name, const minifi::utils::Identifier& uuid) : ProcessorImpl(name, uuid) {}
- explicit DummmyControllerUserProcessor(std::string_view name) : ProcessorImpl(name) {}
static constexpr auto DummyControllerService = core::PropertyDefinitionBuilder<>::createProperty("Dummy Controller Service")
.withDescription("Dummy Controller Service")
.withAllowedTypes<DummyController>()
@@ -88,7 +86,7 @@ class DummmyControllerUserProcessor : public minifi::core::ProcessorImpl {
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& /*session_factory*/) override {
if (auto controller_service = context.getProperty(DummmyControllerUserProcessor::DummyControllerService)) {
- if (!std::dynamic_pointer_cast<DummyController>(context.getControllerService(*controller_service, uuid_))) {
+ if (!std::dynamic_pointer_cast<DummyController>(context.getControllerService(*controller_service, getUUID()))) {
throw minifi::Exception(minifi::ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "Invalid controller service");
}
} else {
@@ -105,9 +103,6 @@ class DummmyControllerUserProcessor : public minifi::core::ProcessorImpl {
static constexpr core::annotation::Input InputRequirement = core::annotation::Input::INPUT_ALLOWED;
static constexpr bool IsSingleThreaded = false;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
-
- private:
- std::shared_ptr<core::logging::Logger> logger_ = core::logging::LoggerFactory<DummmyControllerUserProcessor>::getLogger(uuid_);
};
REGISTER_RESOURCE(DummmyControllerUserProcessor, Processor);
diff --git a/libminifi/test/integration/C2MetricsTest.cpp b/libminifi/test/integration/C2MetricsTest.cpp
index 7451e0c3b..75519215b 100644
--- a/libminifi/test/integration/C2MetricsTest.cpp
+++ b/libminifi/test/integration/C2MetricsTest.cpp
@@ -58,6 +58,8 @@ class VerifyC2Metrics : public VerifyC2Base {
const std::atomic_bool& metrics_updated_successfully_;
};
+#define CHECK_EXPR(expr) (expr ? true : (std::cout << #expr << " failed" << std::endl, false))
+
class MetricsHandler: public HeartbeatHandler {
public:
explicit MetricsHandler(std::atomic_bool& metrics_updated_successfully, std::shared_ptr<minifi::Configure> configuration, const std::filesystem::path& replacement_config_path)
@@ -105,13 +107,13 @@ class MetricsHandler: public HeartbeatHandler {
void verifyMetrics(const rapidjson::Document& root) {
auto initial_metrics_verified =
- root.HasMember("metrics") &&
- root["metrics"].HasMember("RuntimeMetrics") &&
- root["metrics"].HasMember("LoadMetrics") &&
- root["metrics"].HasMember("ProcessorMetrics") &&
- verifyRuntimeMetrics(root["metrics"]["RuntimeMetrics"]) &&
- verifyLoadMetrics(root["metrics"]["LoadMetrics"]) &&
- verifyProcessorMetrics(root["metrics"]["ProcessorMetrics"]);
+ CHECK_EXPR(root.HasMember("metrics")) &&
+ CHECK_EXPR(root["metrics"].HasMember("RuntimeMetrics")) &&
+ CHECK_EXPR(root["metrics"].HasMember("LoadMetrics")) &&
+ CHECK_EXPR(root["metrics"].HasMember("ProcessorMetrics")) &&
+ CHECK_EXPR(verifyRuntimeMetrics(root["metrics"]["RuntimeMetrics"])) &&
+ CHECK_EXPR(verifyLoadMetrics(root["metrics"]["LoadMetrics"])) &&
+ CHECK_EXPR(verifyProcessorMetrics(root["metrics"]["ProcessorMetrics"]));
if (initial_metrics_verified) {
test_state_ = TestState::SEND_NEW_CONFIG;
}
@@ -163,40 +165,42 @@ class MetricsHandler: public HeartbeatHandler {
static bool verifyProcessorBulletins(const rapidjson::Value& runtime_metrics) {
if (!runtime_metrics["flowInfo"].HasMember("processorBulletins")) {
+ std::cout << "!runtime_metrics[\"flowInfo\"].HasMember(\"processorBulletins\")" << std::endl;
return false;
}
auto bulletins = runtime_metrics["flowInfo"]["processorBulletins"].GetArray();
return std::any_of(bulletins.begin(), bulletins.end(), [](const auto& bulletin) {
std::string message = bulletin["message"].GetString();
- return bulletin["id"].GetInt() > 0 &&
- bulletin["timestamp"].GetInt64() > 0 &&
- bulletin["level"].GetString() == std::string("ERROR") &&
- bulletin["category"].GetString() == std::string("Log Message") &&
- message.find("Error connecting to") != std::string::npos &&
- message.find(GETTCP_UUID) != std::string::npos &&
- bulletin["groupId"].GetString() == std::string(PROCESS_GROUP_UUID) &&
- bulletin["groupName"].GetString() == std::string("MiNiFi Flow") &&
- bulletin["groupPath"].GetString() == std::string("MiNiFi Flow") &&
- bulletin["sourceId"].GetString() == std::string(GETTCP_UUID) &&
- bulletin["sourceName"].GetString() == std::string("GetTCP");
+ return CHECK_EXPR(bulletin["id"].GetInt() > 0) &&
+ CHECK_EXPR(bulletin["timestamp"].GetInt64() > 0) &&
+ CHECK_EXPR(bulletin["level"].GetString() == std::string("ERROR")) &&
+ CHECK_EXPR(bulletin["category"].GetString() == std::string("Log Message")) &&
+ CHECK_EXPR(message.find("Error connecting to") != std::string::npos) &&
+ CHECK_EXPR(message.find(GETTCP_UUID) != std::string::npos) &&
+ CHECK_EXPR(bulletin["groupId"].GetString() == std::string(PROCESS_GROUP_UUID)) &&
+ CHECK_EXPR(bulletin["groupName"].GetString() == std::string("MiNiFi Flow")) &&
+ CHECK_EXPR(bulletin["groupPath"].GetString() == std::string("MiNiFi Flow")) &&
+ CHECK_EXPR(bulletin["sourceId"].GetString() == std::string(GETTCP_UUID)) &&
+ CHECK_EXPR(bulletin["sourceName"].GetString() == std::string("GetTCP"));
});
}
static bool verifyRuntimeMetrics(const rapidjson::Value& runtime_metrics) {
- return verifyCommonRuntimeMetricNodes(runtime_metrics, "2438e3c8-015a-1000-79ca-83af40ec1997") &&
+ return CHECK_EXPR(verifyCommonRuntimeMetricNodes(runtime_metrics, "2438e3c8-015a-1000-79ca-83af40ec1997")) &&
[&]() {
const auto processor_statuses = runtime_metrics["flowInfo"]["processorStatuses"].GetArray();
if (processor_statuses.Size() != 2) {
+ std::cout << "processor_statuses.Size() != 2" << std::endl;
return false;
}
return std::all_of(processor_statuses.begin(), processor_statuses.end(), [&](const auto& processor) {
if (processor["id"].GetString() != std::string(GETTCP_UUID) && processor["id"].GetString() != std::string(LOGATTRIBUTE1_UUID)) {
throw std::runtime_error(std::string("Unexpected processor id in processorStatuses: ") + processor["id"].GetString());
}
- return processorMetricsAreValid(processor);
+ return CHECK_EXPR(processorMetricsAreValid(processor));
});
}() &&
- verifyProcessorBulletins(runtime_metrics);
+ CHECK_EXPR(verifyProcessorBulletins(runtime_metrics));
}
static bool verifyUpdatedRuntimeMetrics(const rapidjson::Value& runtime_metrics) {
diff --git a/libminifi/test/unit/BulletinStoreTests.cpp b/libminifi/test/unit/BulletinStoreTests.cpp
index deb811725..155b441bc 100644
--- a/libminifi/test/unit/BulletinStoreTests.cpp
+++ b/libminifi/test/unit/BulletinStoreTests.cpp
@@ -22,6 +22,7 @@
#include "core/BulletinStore.h"
#include "properties/Configure.h"
#include "unit/DummyProcessor.h"
+#include "unit/ProcessorUtils.h"
using namespace std::literals::chrono_literals;
@@ -35,7 +36,7 @@ class BulletinStoreTestAccessor {
};
std::unique_ptr<core::Processor> createDummyProcessor() {
- auto processor = std::make_unique<DummyProcessor>("DummyProcessor", minifi::utils::Identifier::parse("4d7fa7e6-2459-46dd-b2ba-61517239edf5").value());
+ auto processor = test::utils::make_processor<DummyProcessor>("DummyProcessor", minifi::utils::Identifier::parse("4d7fa7e6-2459-46dd-b2ba-61517239edf5").value());
processor->setProcessGroupUUIDStr("68fa9ae4-b9fc-4873-b0d9-edab59fdb0c2");
processor->setProcessGroupName("sub_group");
processor->setProcessGroupPath("root / sub_group");
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContext.h b/minifi-api/include/minifi-cpp/core/ProcessContext.h
index f4f963e1d..fdc04a874 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContext.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContext.h
@@ -74,6 +74,8 @@ class ProcessContext : public virtual core::VariableRegistry, public virtual uti
virtual nonstd::expected<std::string, std::error_code> getDynamicProperty(std::string_view name, const FlowFile* flow_file = nullptr) const = 0;
virtual nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value) = 0;
+ virtual nonstd::expected<std::string, std::error_code> getRawDynamicProperty(std::string_view name) const = 0;
+
virtual std::vector<std::string> getDynamicPropertyKeys() const = 0;
virtual std::map<std::string, std::string> getDynamicProperties(const FlowFile* flow_file = nullptr) const = 0;
diff --git a/minifi-api/include/minifi-cpp/core/ProcessorApi.h b/minifi-api/include/minifi-cpp/core/ProcessorApi.h
index e9cfda446..0b12a4646 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessorApi.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessorApi.h
@@ -68,6 +68,7 @@ class ProcessorApi {
virtual void notifyStop() = 0;
virtual annotation::Input getInputRequirement() const = 0;
virtual gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const = 0;
+ virtual void setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& callback) = 0;
};
} // namespace core
diff --git a/utils/include/core/ProcessContext.h b/utils/include/core/ProcessContext.h
index df73d4cc5..e3b4c1d8d 100644
--- a/utils/include/core/ProcessContext.h
+++ b/utils/include/core/ProcessContext.h
@@ -71,6 +71,7 @@ class ProcessContextImpl : public core::VariableRegistryImpl, public virtual Pro
nonstd::expected<std::string, std::error_code> getDynamicProperty(std::string_view name, const FlowFile*) const override;
nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value) override;
nonstd::expected<std::string, std::error_code> getRawProperty(std::string_view name) const override;
+ nonstd::expected<std::string, std::error_code> getRawDynamicProperty(std::string_view name) const override;
[[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> getAllPropertyValues(std::string_view name) const override;
bool hasIncomingConnections() const override;
diff --git a/utils/include/core/ProcessorImpl.h b/utils/include/core/ProcessorImpl.h
index 23ba2f516..6d51b70ff 100644
--- a/utils/include/core/ProcessorImpl.h
+++ b/utils/include/core/ProcessorImpl.h
@@ -130,6 +130,8 @@ class ProcessorImpl : public virtual ProcessorApi {
void restore(const std::shared_ptr<FlowFile>& file) override;
+ void setLoggerCallback(const std::function<void(logging::LOG_LEVEL level, const std::string& message)>& /*callback*/) override;
+
std::string getName() const;
utils::Identifier getUUID() const;
utils::SmallString<36> getUUIDStr() const;
(nifi-minifi-cpp) 11/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 13460b68551bc2903c18a002f2d0ef1a00e2c4a0
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Apr 28 12:48:16 2025 +0200
Rebase fix
---
extensions/procfs/tests/ProcFsMonitorTests.cpp | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/extensions/procfs/tests/ProcFsMonitorTests.cpp b/extensions/procfs/tests/ProcFsMonitorTests.cpp
index cf2ccb4eb..55d4bcfb3 100644
--- a/extensions/procfs/tests/ProcFsMonitorTests.cpp
+++ b/extensions/procfs/tests/ProcFsMonitorTests.cpp
@@ -19,11 +19,12 @@
#include "unit/SingleProcessorTestController.h"
#include "unit/Catch.h"
#include "processors/ProcFsMonitor.h"
+#include "unit/ProcessorUtils.h"
namespace org::apache::nifi::minifi::extensions::procfs::tests {
TEST_CASE("ProcFsMonitorTests", "[procfsmonitortests]") {
- org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{std::make_unique<ProcFsMonitor>("ProcFsMonitor")};
+ org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{minifi::test::utils::make_processor<ProcFsMonitor>("ProcFsMonitor")};
auto proc_fs_monitor = test_controller_.getProcessor();
SECTION("Absolute JSON") {
(nifi-minifi-cpp) 23/37: Review changes, upgrade concurrentqueue
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit bf1134e5a1e762211fe99ec33b425abfcce32cfb
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Tue Jun 24 11:45:55 2025 +0200
Review changes, upgrade concurrentqueue
---
core-framework/include/core/ProcessorImpl.h | 6 +-
core-framework/src/core/ProcessorImpl.cpp | 4 +-
core-framework/src/io/OutputStream.cpp | 12 +-
extensions/aws/processors/AwsProcessor.h | 3 +-
extensions/aws/processors/DeleteS3Object.h | 2 +-
extensions/aws/processors/FetchS3Object.h | 2 +-
extensions/aws/processors/ListS3.h | 2 +-
extensions/aws/processors/PutKinesisStream.h | 2 +-
extensions/aws/processors/PutS3Object.h | 2 +-
extensions/aws/processors/S3Processor.cpp | 2 +-
extensions/aws/processors/S3Processor.h | 1 +
extensions/aws/tests/PutKinesisStreamTests.cpp | 2 +-
extensions/python/ExecutePythonProcessor.cpp | 1 -
thirdparty/concurrentqueue/concurrentqueue.h | 1370 +++++++++++++-----------
14 files changed, 764 insertions(+), 647 deletions(-)
diff --git a/core-framework/include/core/ProcessorImpl.h b/core-framework/include/core/ProcessorImpl.h
index 6d51b70ff..6b14e5e3d 100644
--- a/core-framework/include/core/ProcessorImpl.h
+++ b/core-framework/include/core/ProcessorImpl.h
@@ -77,8 +77,10 @@ class ProcessorImpl : public virtual ProcessorApi {
public:
explicit ProcessorImpl(ProcessorMetadata info);
- ProcessorImpl(const ProcessorImpl& parent) = delete;
- ProcessorImpl& operator=(const ProcessorImpl& parent) = delete;
+ ProcessorImpl(const ProcessorImpl&) = delete;
+ ProcessorImpl(ProcessorImpl&&) = delete;
+ ProcessorImpl& operator=(const ProcessorImpl&) = delete;
+ ProcessorImpl& operator=(ProcessorImpl&&) = delete;
~ProcessorImpl() override;
diff --git a/core-framework/src/core/ProcessorImpl.cpp b/core-framework/src/core/ProcessorImpl.cpp
index 460ba47ff..68f5885bd 100644
--- a/core-framework/src/core/ProcessorImpl.cpp
+++ b/core-framework/src/core/ProcessorImpl.cpp
@@ -45,10 +45,10 @@ using namespace std::literals::chrono_literals;
namespace org::apache::nifi::minifi::core {
ProcessorImpl::ProcessorImpl(ProcessorMetadata info)
- : info_(info),
+ : info_(std::move(info)),
trigger_when_empty_(false),
metrics_(std::make_shared<ProcessorMetricsImpl>(*this)),
- logger_(info.logger) {
+ logger_(info_.logger) {
logger_->log_debug("Processor {} created with uuid {}", getName(), getUUIDStr());
}
diff --git a/core-framework/src/io/OutputStream.cpp b/core-framework/src/io/OutputStream.cpp
index c19fff602..1b48912c8 100644
--- a/core-framework/src/io/OutputStream.cpp
+++ b/core-framework/src/io/OutputStream.cpp
@@ -24,11 +24,7 @@
#include "io/OutputStream.h"
#include "utils/gsl.h"
-namespace org {
-namespace apache {
-namespace nifi {
-namespace minifi {
-namespace io {
+namespace org::apache::nifi::minifi::io {
size_t OutputStream::write(const std::vector<uint8_t>& buffer, size_t len) {
if (buffer.size() < len) {
@@ -77,8 +73,4 @@ size_t OutputStream::write_str(const char* str, uint32_t len, bool widen) {
return ret + write(reinterpret_cast<const uint8_t *>(str), len);
}
-} /* namespace io */
-} /* namespace minifi */
-} /* namespace nifi */
-} /* namespace apache */
-} /* namespace org */
+} // namespace org::apache::nifi::minifi::io
diff --git a/extensions/aws/processors/AwsProcessor.h b/extensions/aws/processors/AwsProcessor.h
index 542d0c75b..f8ed99608 100644
--- a/extensions/aws/processors/AwsProcessor.h
+++ b/extensions/aws/processors/AwsProcessor.h
@@ -94,7 +94,7 @@ struct CommonProperties {
std::string endpoint_override_url;
};
-class AwsProcessor : public core::ProcessorImpl {
+class AwsProcessor : public core::ProcessorImpl { // NOLINT(cppcoreguidelines-special-member-functions)
public:
EXTENSIONAPI static constexpr auto AccessKey = core::PropertyDefinitionBuilder<>::createProperty("Access Key")
.withDescription("AWS account access key")
@@ -170,6 +170,7 @@ class AwsProcessor : public core::ProcessorImpl {
});
using ProcessorImpl::ProcessorImpl;
+ ~AwsProcessor() override = default;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
diff --git a/extensions/aws/processors/DeleteS3Object.h b/extensions/aws/processors/DeleteS3Object.h
index 6a2047755..f723c58fd 100644
--- a/extensions/aws/processors/DeleteS3Object.h
+++ b/extensions/aws/processors/DeleteS3Object.h
@@ -35,7 +35,7 @@ class FlowProcessorS3TestsFixture;
namespace org::apache::nifi::minifi::aws::processors {
-class DeleteS3Object : public S3Processor {
+class DeleteS3Object : public S3Processor { // NOLINT(cppcoreguidelines-special-member-functions)
public:
EXTENSIONAPI static constexpr const char* Description = "Deletes FlowFiles on an Amazon S3 Bucket. If attempting to delete a file that does not exist, FlowFile is routed to success.";
diff --git a/extensions/aws/processors/FetchS3Object.h b/extensions/aws/processors/FetchS3Object.h
index fa5fd182f..19d9529bf 100644
--- a/extensions/aws/processors/FetchS3Object.h
+++ b/extensions/aws/processors/FetchS3Object.h
@@ -37,7 +37,7 @@ class FlowProcessorS3TestsFixture;
namespace org::apache::nifi::minifi::aws::processors {
-class FetchS3Object : public S3Processor {
+class FetchS3Object : public S3Processor { // NOLINT(cppcoreguidelines-special-member-functions)
public:
EXTENSIONAPI static constexpr const char* Description = "This Processor retrieves the contents of an S3 Object and writes it to the content of a FlowFile.";
diff --git a/extensions/aws/processors/ListS3.h b/extensions/aws/processors/ListS3.h
index b3e457952..4c9d64fb1 100644
--- a/extensions/aws/processors/ListS3.h
+++ b/extensions/aws/processors/ListS3.h
@@ -32,7 +32,7 @@
namespace org::apache::nifi::minifi::aws::processors {
-class ListS3 : public S3Processor {
+class ListS3 : public S3Processor { // NOLINT(cppcoreguidelines-special-member-functions)
public:
EXTENSIONAPI static constexpr const char* Description = "This Processor retrieves a listing of objects from an Amazon S3 bucket.";
diff --git a/extensions/aws/processors/PutKinesisStream.h b/extensions/aws/processors/PutKinesisStream.h
index 73d308dcf..458d52907 100644
--- a/extensions/aws/processors/PutKinesisStream.h
+++ b/extensions/aws/processors/PutKinesisStream.h
@@ -33,7 +33,7 @@
namespace org::apache::nifi::minifi::aws::processors {
-class PutKinesisStream : public AwsProcessor {
+class PutKinesisStream : public AwsProcessor { // NOLINT(cppcoreguidelines-special-member-functions)
public:
EXTENSIONAPI static constexpr const char* Description = "Sends the contents to a specified Amazon Kinesis. In order to send data to Kinesis, the stream name has to be specified.";
diff --git a/extensions/aws/processors/PutS3Object.h b/extensions/aws/processors/PutS3Object.h
index 0a5cbfe4e..03647b5da 100644
--- a/extensions/aws/processors/PutS3Object.h
+++ b/extensions/aws/processors/PutS3Object.h
@@ -43,7 +43,7 @@ class FlowProcessorS3TestsFixture;
namespace org::apache::nifi::minifi::aws::processors {
-class PutS3Object : public S3Processor {
+class PutS3Object : public S3Processor { // NOLINT(cppcoreguidelines-special-member-functions)
public:
static constexpr auto CANNED_ACLS = minifi::utils::getKeys(minifi::aws::s3::CANNED_ACL_MAP);
static constexpr auto STORAGE_CLASSES = minifi::utils::getKeys(minifi::aws::s3::STORAGE_CLASS_MAP);
diff --git a/extensions/aws/processors/S3Processor.cpp b/extensions/aws/processors/S3Processor.cpp
index 6bb6402f4..532bb6012 100644
--- a/extensions/aws/processors/S3Processor.cpp
+++ b/extensions/aws/processors/S3Processor.cpp
@@ -33,7 +33,7 @@
namespace org::apache::nifi::minifi::aws::processors {
S3Processor::S3Processor(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : AwsProcessor(info),
+ : AwsProcessor(std::move(info)),
s3_wrapper_(std::move(s3_request_sender)) {
}
diff --git a/extensions/aws/processors/S3Processor.h b/extensions/aws/processors/S3Processor.h
index cdcee4c4c..a5f966b5d 100644
--- a/extensions/aws/processors/S3Processor.h
+++ b/extensions/aws/processors/S3Processor.h
@@ -51,6 +51,7 @@ class S3Processor : public AwsProcessor {
EXTENSIONAPI static constexpr auto Properties = minifi::utils::array_cat(AwsProcessor::Properties, std::to_array<core::PropertyReference>({Bucket}));
using AwsProcessor::AwsProcessor;
+ ~S3Processor() override = default;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
diff --git a/extensions/aws/tests/PutKinesisStreamTests.cpp b/extensions/aws/tests/PutKinesisStreamTests.cpp
index b6532b5b3..3a3d4c105 100644
--- a/extensions/aws/tests/PutKinesisStreamTests.cpp
+++ b/extensions/aws/tests/PutKinesisStreamTests.cpp
@@ -83,7 +83,7 @@ class MockKinesisClient final : public Aws::Kinesis::KinesisClient {
mutable uint32_t sequence_number_ = 0;
};
-class PutKinesisStreamMocked final : public aws::processors::PutKinesisStream {
+class PutKinesisStreamMocked final : public aws::processors::PutKinesisStream { // NOLINT(cppcoreguidelines-special-member-functions)
public:
static constexpr const char* Description = "PutKinesisStreamMocked";
diff --git a/extensions/python/ExecutePythonProcessor.cpp b/extensions/python/ExecutePythonProcessor.cpp
index 7daec15c1..48a58ad6f 100644
--- a/extensions/python/ExecutePythonProcessor.cpp
+++ b/extensions/python/ExecutePythonProcessor.cpp
@@ -142,7 +142,6 @@ void ExecutePythonProcessor::loadScript() {
}
loadScriptFromFile();
last_script_write_time_ = utils::file::last_write_time(script_file_path_);
- return;
}
}
diff --git a/thirdparty/concurrentqueue/concurrentqueue.h b/thirdparty/concurrentqueue/concurrentqueue.h
index db852d12c..b9c0bc7a8 100644
--- a/thirdparty/concurrentqueue/concurrentqueue.h
+++ b/thirdparty/concurrentqueue/concurrentqueue.h
@@ -5,7 +5,7 @@
// http://moodycamel.com/blog/2014/detailed-design-of-a-lock-free-queue
// Simplified BSD license:
-// Copyright (c) 2013-2016, Cameron Desrochers.
+// Copyright (c) 2013-2020, Cameron Desrochers.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without modification,
@@ -27,9 +27,11 @@
// TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
// EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+// Also dual-licensed under the Boost Software License (see LICENSE.md)
+
#pragma once
-#if defined(__GNUC__)
+#if defined(__GNUC__) && !defined(__INTEL_COMPILER)
// Disable -Wconversion warnings (spuriously triggered when Traits::size_t and
// Traits::index_t are set to < 32 bits, causing integer promotion, causing warnings
// upon assigning any computed values)
@@ -41,6 +43,13 @@
#endif
#endif
+#if defined(_MSC_VER) && (!defined(_HAS_CXX17) || !_HAS_CXX17)
+// VS2019 with /W4 warns about constant conditional expressions but unless /std=c++17 or higher
+// does not support `if constexpr`, so we have no choice but to simply disable the warning
+#pragma warning(push)
+#pragma warning(disable: 4127) // conditional expression is constant
+#endif
+
#if defined(__APPLE__)
#include "TargetConditionals.h"
#endif
@@ -68,6 +77,7 @@
#include <climits> // for CHAR_BIT
#include <array>
#include <thread> // partly for __WINPTHREADS_VERSION if on MinGW-w64 w/ POSIX threading
+#include <mutex> // used for thread exit synchronization
// Platform-specific definitions of a numeric thread ID type and an invalid value
namespace moodycamel { namespace details {
@@ -95,10 +105,10 @@ namespace moodycamel { namespace details {
static const thread_id_t invalid_thread_id2 = 0xFFFFFFFFU; // Not technically guaranteed to be invalid, but is never used in practice. Note that all Win32 thread IDs are presently multiples of 4.
static inline thread_id_t thread_id() { return static_cast<thread_id_t>(::GetCurrentThreadId()); }
} }
-#elif defined(__arm__) || defined(_M_ARM) || defined(__aarch64__) || (defined(__APPLE__) && TARGET_OS_IPHONE)
+#elif defined(__arm__) || defined(_M_ARM) || defined(__aarch64__) || (defined(__APPLE__) && TARGET_OS_IPHONE) || defined(__MVS__) || defined(MOODYCAMEL_NO_THREAD_LOCAL)
namespace moodycamel { namespace details {
static_assert(sizeof(std::thread::id) == 4 || sizeof(std::thread::id) == 8, "std::thread::id is expected to be either 4 or 8 bytes");
-
+
typedef std::thread::id thread_id_t;
static const thread_id_t invalid_thread_id; // Default ctor creates invalid ID
@@ -145,10 +155,21 @@ namespace moodycamel { namespace details {
typedef std::uintptr_t thread_id_t;
static const thread_id_t invalid_thread_id = 0; // Address can't be nullptr
static const thread_id_t invalid_thread_id2 = 1; // Member accesses off a null pointer are also generally invalid. Plus it's not aligned.
- static inline thread_id_t thread_id() { static MOODYCAMEL_THREADLOCAL int x; return reinterpret_cast<thread_id_t>(&x); }
+ inline thread_id_t thread_id() { static MOODYCAMEL_THREADLOCAL int x; return reinterpret_cast<thread_id_t>(&x); }
} }
#endif
+// Constexpr if
+#ifndef MOODYCAMEL_CONSTEXPR_IF
+#if (defined(_MSC_VER) && defined(_HAS_CXX17) && _HAS_CXX17) || __cplusplus > 201402L
+#define MOODYCAMEL_CONSTEXPR_IF if constexpr
+#define MOODYCAMEL_MAYBE_UNUSED [[maybe_unused]]
+#else
+#define MOODYCAMEL_CONSTEXPR_IF if
+#define MOODYCAMEL_MAYBE_UNUSED
+#endif
+#endif
+
// Exceptions
#ifndef MOODYCAMEL_EXCEPTIONS_ENABLED
#if (defined(_MSC_VER) && defined(_CPPUNWIND)) || (defined(__GNUC__) && defined(__EXCEPTIONS)) || (!defined(_MSC_VER) && !defined(__GNUC__))
@@ -161,8 +182,8 @@ namespace moodycamel { namespace details {
#define MOODYCAMEL_RETHROW throw
#define MOODYCAMEL_THROW(expr) throw (expr)
#else
-#define MOODYCAMEL_TRY if (true)
-#define MOODYCAMEL_CATCH(...) else if (false)
+#define MOODYCAMEL_TRY MOODYCAMEL_CONSTEXPR_IF (true)
+#define MOODYCAMEL_CATCH(...) else MOODYCAMEL_CONSTEXPR_IF (false)
#define MOODYCAMEL_RETHROW
#define MOODYCAMEL_THROW(expr)
#endif
@@ -196,14 +217,14 @@ namespace moodycamel { namespace details {
// VS2013 doesn't support `thread_local`, and MinGW-w64 w/ POSIX threading has a crippling bug: http://sourceforge.net/p/mingw-w64/bugs/445
// g++ <=4.7 doesn't support thread_local either.
// Finally, iOS/ARM doesn't have support for it either, and g++/ARM allows it to compile but it's unconfirmed to actually work
-#if (!defined(_MSC_VER) || _MSC_VER >= 1900) && (!defined(__MINGW32__) && !defined(__MINGW64__) || !defined(__WINPTHREADS_VERSION)) && (!defined(__GNUC__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)) && (!defined(__APPLE__) || !TARGET_OS_IPHONE) && !defined(__arm__) && !defined(_M_ARM) && !defined(__aarch64__)
+#if (!defined(_MSC_VER) || _MSC_VER >= 1900) && (!defined(__MINGW32__) && !defined(__MINGW64__) || !defined(__WINPTHREADS_VERSION)) && (!defined(__GNUC__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)) && (!defined(__APPLE__) || !TARGET_OS_IPHONE) && !defined(__arm__) && !defined(_M_ARM) && !defined(__aarch64__) && !defined(__MVS__)
// Assume `thread_local` is fully supported in all other C++11 compilers/platforms
-//#define MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED // always disabled for now since several users report having problems with it on
+#define MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED // tentatively enabled for now; years ago several users report having problems with it on
#endif
#endif
#endif
-// VS2012 doesn't support deleted functions.
+// VS2012 doesn't support deleted functions.
// In this case, we declare the function normally but don't define it. A link error will be generated if the function is called.
#ifndef MOODYCAMEL_DELETE_FUNCTION
#if defined(_MSC_VER) && _MSC_VER < 1800
@@ -213,14 +234,52 @@ namespace moodycamel { namespace details {
#endif
#endif
+namespace moodycamel { namespace details {
+#ifndef MOODYCAMEL_ALIGNAS
+// VS2013 doesn't support alignas or alignof, and align() requires a constant literal
+#if defined(_MSC_VER) && _MSC_VER <= 1800
+#define MOODYCAMEL_ALIGNAS(alignment) __declspec(align(alignment))
+#define MOODYCAMEL_ALIGNOF(obj) __alignof(obj)
+#define MOODYCAMEL_ALIGNED_TYPE_LIKE(T, obj) typename details::Vs2013Aligned<std::alignment_of<obj>::value, T>::type
+ template<int Align, typename T> struct Vs2013Aligned { }; // default, unsupported alignment
+ template<typename T> struct Vs2013Aligned<1, T> { typedef __declspec(align(1)) T type; };
+ template<typename T> struct Vs2013Aligned<2, T> { typedef __declspec(align(2)) T type; };
+ template<typename T> struct Vs2013Aligned<4, T> { typedef __declspec(align(4)) T type; };
+ template<typename T> struct Vs2013Aligned<8, T> { typedef __declspec(align(8)) T type; };
+ template<typename T> struct Vs2013Aligned<16, T> { typedef __declspec(align(16)) T type; };
+ template<typename T> struct Vs2013Aligned<32, T> { typedef __declspec(align(32)) T type; };
+ template<typename T> struct Vs2013Aligned<64, T> { typedef __declspec(align(64)) T type; };
+ template<typename T> struct Vs2013Aligned<128, T> { typedef __declspec(align(128)) T type; };
+ template<typename T> struct Vs2013Aligned<256, T> { typedef __declspec(align(256)) T type; };
+#else
+ template<typename T> struct identity { typedef T type; };
+#define MOODYCAMEL_ALIGNAS(alignment) alignas(alignment)
+#define MOODYCAMEL_ALIGNOF(obj) alignof(obj)
+#define MOODYCAMEL_ALIGNED_TYPE_LIKE(T, obj) alignas(alignof(obj)) typename details::identity<T>::type
+#endif
+#endif
+} }
+
+
+// TSAN can false report races in lock-free code. To enable TSAN to be used from projects that use this one,
+// we can apply per-function compile-time suppression.
+// See https://clang.llvm.org/docs/ThreadSanitizer.html#has-feature-thread-sanitizer
+#define MOODYCAMEL_NO_TSAN
+#if defined(__has_feature)
+ #if __has_feature(thread_sanitizer)
+ #undef MOODYCAMEL_NO_TSAN
+ #define MOODYCAMEL_NO_TSAN __attribute__((no_sanitize("thread")))
+ #endif // TSAN
+#endif // TSAN
+
// Compiler-specific likely/unlikely hints
namespace moodycamel { namespace details {
#if defined(__GNUC__)
- inline bool likely(bool x) { return __builtin_expect((x), true); }
- inline bool unlikely(bool x) { return __builtin_expect((x), false); }
+ static inline bool (likely)(bool x) { return __builtin_expect((x), true); }
+ static inline bool (unlikely)(bool x) { return __builtin_expect((x), false); }
#else
- inline bool likely(bool x) { return x; }
- inline bool unlikely(bool x) { return x; }
+ static inline bool (likely)(bool x) { return x; }
+ static inline bool (unlikely)(bool x) { return x; }
#endif
} }
@@ -238,11 +297,19 @@ namespace details {
: static_cast<T>(-1);
};
-#if defined(__GNUC__) && !defined( __clang__ )
- typedef ::max_align_t max_align_t; // GCC forgot to add it to std:: for a while
+#if defined(__GLIBCXX__)
+ typedef ::max_align_t std_max_align_t; // libstdc++ forgot to add it to std:: for a while
#else
- typedef std::max_align_t max_align_t; // Others (e.g. MSVC) insist it can *only* be accessed via std::
+ typedef std::max_align_t std_max_align_t; // Others (e.g. MSVC) insist it can *only* be accessed via std::
#endif
+
+ // Some platforms have incorrectly set max_align_t to a type with <8 bytes alignment even while supporting
+ // 8-byte aligned scalar values (*cough* 32-bit iOS). Work around this with our own union. See issue #64.
+ typedef union {
+ std_max_align_t x;
+ long long y;
+ void* z;
+ } max_align_t;
}
// Default traits for the ConcurrentQueue. To change some of the
@@ -255,7 +322,7 @@ struct ConcurrentQueueDefaultTraits
{
// General-purpose size type. std::size_t is strongly recommended.
typedef std::size_t size_t;
-
+
// The type used for the enqueue and dequeue indices. Must be at least as
// large as size_t. Should be significantly larger than the number of elements
// you expect to hold at once, especially if you have a high turnover rate;
@@ -267,47 +334,60 @@ struct ConcurrentQueueDefaultTraits
// whether the queue is lock-free with a 64-int type depends on the whether
// std::atomic<std::uint64_t> is lock-free, which is platform-specific.
typedef std::size_t index_t;
-
+
// Internally, all elements are enqueued and dequeued from multi-element
// blocks; this is the smallest controllable unit. If you expect few elements
// but many producers, a smaller block size should be favoured. For few producers
// and/or many elements, a larger block size is preferred. A sane default
// is provided. Must be a power of 2.
static const size_t BLOCK_SIZE = 32;
-
+
// For explicit producers (i.e. when using a producer token), the block is
// checked for being empty by iterating through a list of flags, one per element.
// For large block sizes, this is too inefficient, and switching to an atomic
// counter-based approach is faster. The switch is made for block sizes strictly
// larger than this threshold.
static const size_t EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD = 32;
-
+
// How many full blocks can be expected for a single explicit producer? This should
// reflect that number's maximum for optimal performance. Must be a power of 2.
static const size_t EXPLICIT_INITIAL_INDEX_SIZE = 32;
-
+
// How many full blocks can be expected for a single implicit producer? This should
// reflect that number's maximum for optimal performance. Must be a power of 2.
static const size_t IMPLICIT_INITIAL_INDEX_SIZE = 32;
-
+
// The initial size of the hash table mapping thread IDs to implicit producers.
// Note that the hash is resized every time it becomes half full.
// Must be a power of two, and either 0 or at least 1. If 0, implicit production
// (using the enqueue methods without an explicit producer token) is disabled.
static const size_t INITIAL_IMPLICIT_PRODUCER_HASH_SIZE = 32;
-
+
// Controls the number of items that an explicit consumer (i.e. one with a token)
// must consume before it causes all consumers to rotate and move on to the next
// internal queue.
static const std::uint32_t EXPLICIT_CONSUMER_CONSUMPTION_QUOTA_BEFORE_ROTATE = 256;
-
+
// The maximum number of elements (inclusive) that can be enqueued to a sub-queue.
// Enqueue operations that would cause this limit to be surpassed will fail. Note
// that this limit is enforced at the block level (for performance reasons), i.e.
// it's rounded up to the nearest block size.
static const size_t MAX_SUBQUEUE_SIZE = details::const_numeric_max<size_t>::value;
-
-
+
+ // The number of times to spin before sleeping when waiting on a semaphore.
+ // Recommended values are on the order of 1000-10000 unless the number of
+ // consumer threads exceeds the number of idle cores (in which case try 0-100).
+ // Only affects instances of the BlockingConcurrentQueue.
+ static const int MAX_SEMA_SPINS = 10000;
+
+ // Whether to recycle dynamically-allocated blocks into an internal free list or
+ // not. If false, only pre-allocated blocks (controlled by the constructor
+ // arguments) will be recycled, and all others will be `free`d back to the heap.
+ // Note that blocks consumed by explicit producers are only freed on destruction
+ // of the queue (not following destruction of the token) regardless of this trait.
+ static const bool RECYCLE_ALLOCATED_BLOCKS = false;
+
+
#ifndef MCDBGQ_USE_RELACY
// Memory allocation can be customized if needed.
// malloc should return nullptr on failure, and handle alignment like std::malloc.
@@ -353,13 +433,13 @@ namespace details
ConcurrentQueueProducerTypelessBase* next;
std::atomic<bool> inactive;
ProducerToken* token;
-
+
ConcurrentQueueProducerTypelessBase()
: next(nullptr), inactive(false), token(nullptr)
{
}
};
-
+
template<bool use32> struct _hash_32_or_64 {
static inline std::uint32_t hash(std::uint32_t h)
{
@@ -385,28 +465,23 @@ namespace details
}
};
template<std::size_t size> struct hash_32_or_64 : public _hash_32_or_64<(size > 4)> { };
-
+
static inline size_t hash_thread_id(thread_id_t id)
{
static_assert(sizeof(thread_id_t) <= 8, "Expected a platform where thread IDs are at most 64-bit values");
return static_cast<size_t>(hash_32_or_64<sizeof(thread_id_converter<thread_id_t>::thread_id_hash_t)>::hash(
thread_id_converter<thread_id_t>::prehash(id)));
}
-
+
template<typename T>
static inline bool circular_less_than(T a, T b)
{
-#ifdef _MSC_VER
-#pragma warning(push)
-#pragma warning(disable: 4554)
-#endif
static_assert(std::is_integral<T>::value && !std::numeric_limits<T>::is_signed, "circular_less_than is intended to be used only with unsigned integer types");
- return static_cast<T>(a - b) > static_cast<T>(static_cast<T>(1) << static_cast<T>(sizeof(T) * CHAR_BIT - 1));
-#ifdef _MSC_VER
-#pragma warning(pop)
-#endif
+ return static_cast<T>(a - b) > static_cast<T>(static_cast<T>(1) << (static_cast<T>(sizeof(T) * CHAR_BIT - 1)));
+ // Note: extra parens around rhs of operator<< is MSVC bug: https://developercommunity2.visualstudio.com/t/C4554-triggers-when-both-lhs-and-rhs-is/10034931
+ // silencing the bug requires #pragma warning(disable: 4554) around the calling code and has no effect when done here.
}
-
+
template<typename U>
static inline char* align_for(char* ptr)
{
@@ -430,7 +505,7 @@ namespace details
++x;
return x;
}
-
+
template<typename T>
static inline void swap_relaxed(std::atomic<T>& left, std::atomic<T>& right)
{
@@ -438,13 +513,13 @@ namespace details
left.store(std::move(right.load(std::memory_order_relaxed)), std::memory_order_relaxed);
right.store(std::move(temp), std::memory_order_relaxed);
}
-
+
template<typename T>
static inline T const& nomove(T const& x)
{
return x;
}
-
+
template<bool Enable>
struct nomove_if
{
@@ -454,7 +529,7 @@ namespace details
return x;
}
};
-
+
template<>
struct nomove_if<false>
{
@@ -465,47 +540,56 @@ namespace details
return std::forward<U>(x);
}
};
-
+
template<typename It>
static inline auto deref_noexcept(It& it) MOODYCAMEL_NOEXCEPT -> decltype(*it)
{
return *it;
}
-
+
#if defined(__clang__) || !defined(__GNUC__) || __GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 8)
template<typename T> struct is_trivially_destructible : std::is_trivially_destructible<T> { };
#else
template<typename T> struct is_trivially_destructible : std::has_trivial_destructor<T> { };
#endif
-
+
#ifdef MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED
#ifdef MCDBGQ_USE_RELACY
typedef RelacyThreadExitListener ThreadExitListener;
typedef RelacyThreadExitNotifier ThreadExitNotifier;
#else
+ class ThreadExitNotifier;
+
struct ThreadExitListener
{
typedef void (*callback_t)(void*);
callback_t callback;
void* userData;
-
+
ThreadExitListener* next; // reserved for use by the ThreadExitNotifier
+ ThreadExitNotifier* chain; // reserved for use by the ThreadExitNotifier
};
-
-
+
class ThreadExitNotifier
{
public:
static void subscribe(ThreadExitListener* listener)
{
auto& tlsInst = instance();
+ std::lock_guard<std::mutex> guard(mutex());
listener->next = tlsInst.tail;
+ listener->chain = &tlsInst;
tlsInst.tail = listener;
}
-
+
static void unsubscribe(ThreadExitListener* listener)
{
- auto& tlsInst = instance();
+ std::lock_guard<std::mutex> guard(mutex());
+ if (!listener->chain) {
+ return; // race with ~ThreadExitNotifier
+ }
+ auto& tlsInst = *listener->chain;
+ listener->chain = nullptr;
ThreadExitListener** prev = &tlsInst.tail;
for (auto ptr = tlsInst.tail; ptr != nullptr; ptr = ptr->next) {
if (ptr == listener) {
@@ -515,34 +599,43 @@ namespace details
prev = &ptr->next;
}
}
-
+
private:
ThreadExitNotifier() : tail(nullptr) { }
ThreadExitNotifier(ThreadExitNotifier const&) MOODYCAMEL_DELETE_FUNCTION;
ThreadExitNotifier& operator=(ThreadExitNotifier const&) MOODYCAMEL_DELETE_FUNCTION;
-
+
~ThreadExitNotifier()
{
// This thread is about to exit, let everyone know!
assert(this == &instance() && "If this assert fails, you likely have a buggy compiler! Change the preprocessor conditions such that MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED is no longer defined.");
+ std::lock_guard<std::mutex> guard(mutex());
for (auto ptr = tail; ptr != nullptr; ptr = ptr->next) {
+ ptr->chain = nullptr;
ptr->callback(ptr->userData);
}
}
-
+
// Thread-local
static inline ThreadExitNotifier& instance()
{
static thread_local ThreadExitNotifier notifier;
return notifier;
}
-
+
+ static inline std::mutex& mutex()
+ {
+ // Must be static because the ThreadExitNotifier could be destroyed while unsubscribe is called
+ static std::mutex mutex;
+ return mutex;
+ }
+
private:
ThreadExitListener* tail;
};
#endif
#endif
-
+
template<typename T> struct static_is_lock_free_num { enum { value = 0 }; };
template<> struct static_is_lock_free_num<signed char> { enum { value = ATOMIC_CHAR_LOCK_FREE }; };
template<> struct static_is_lock_free_num<short> { enum { value = ATOMIC_SHORT_LOCK_FREE }; };
@@ -559,10 +652,10 @@ struct ProducerToken
{
template<typename T, typename Traits>
explicit ProducerToken(ConcurrentQueue<T, Traits>& queue);
-
+
template<typename T, typename Traits>
explicit ProducerToken(BlockingConcurrentQueue<T, Traits>& queue);
-
+
ProducerToken(ProducerToken&& other) MOODYCAMEL_NOEXCEPT
: producer(other.producer)
{
@@ -571,13 +664,13 @@ struct ProducerToken
producer->token = this;
}
}
-
+
inline ProducerToken& operator=(ProducerToken&& other) MOODYCAMEL_NOEXCEPT
{
swap(other);
return *this;
}
-
+
void swap(ProducerToken& other) MOODYCAMEL_NOEXCEPT
{
std::swap(producer, other.producer);
@@ -588,7 +681,7 @@ struct ProducerToken
other.producer->token = &other;
}
}
-
+
// A token is always valid unless:
// 1) Memory allocation failed during construction
// 2) It was moved via the move constructor
@@ -598,7 +691,7 @@ struct ProducerToken
// that the token is valid for use with a specific queue,
// but not which one; that's up to the user to track.
inline bool valid() const { return producer != nullptr; }
-
+
~ProducerToken()
{
if (producer != nullptr) {
@@ -606,15 +699,15 @@ struct ProducerToken
producer->inactive.store(true, std::memory_order_release);
}
}
-
+
// Disable copying and assignment
ProducerToken(ProducerToken const&) MOODYCAMEL_DELETE_FUNCTION;
ProducerToken& operator=(ProducerToken const&) MOODYCAMEL_DELETE_FUNCTION;
-
+
private:
template<typename T, typename Traits> friend class ConcurrentQueue;
friend class ConcurrentQueueTests;
-
+
protected:
details::ConcurrentQueueProducerTypelessBase* producer;
};
@@ -624,21 +717,21 @@ struct ConsumerToken
{
template<typename T, typename Traits>
explicit ConsumerToken(ConcurrentQueue<T, Traits>& q);
-
+
template<typename T, typename Traits>
explicit ConsumerToken(BlockingConcurrentQueue<T, Traits>& q);
-
+
ConsumerToken(ConsumerToken&& other) MOODYCAMEL_NOEXCEPT
: initialOffset(other.initialOffset), lastKnownGlobalOffset(other.lastKnownGlobalOffset), itemsConsumedFromCurrent(other.itemsConsumedFromCurrent), currentProducer(other.currentProducer), desiredProducer(other.desiredProducer)
{
}
-
+
inline ConsumerToken& operator=(ConsumerToken&& other) MOODYCAMEL_NOEXCEPT
{
swap(other);
return *this;
}
-
+
void swap(ConsumerToken& other) MOODYCAMEL_NOEXCEPT
{
std::swap(initialOffset, other.initialOffset);
@@ -647,7 +740,7 @@ struct ConsumerToken
std::swap(currentProducer, other.currentProducer);
std::swap(desiredProducer, other.desiredProducer);
}
-
+
// Disable copying and assignment
ConsumerToken(ConsumerToken const&) MOODYCAMEL_DELETE_FUNCTION;
ConsumerToken& operator=(ConsumerToken const&) MOODYCAMEL_DELETE_FUNCTION;
@@ -655,7 +748,7 @@ struct ConsumerToken
private:
template<typename T, typename Traits> friend class ConcurrentQueue;
friend class ConcurrentQueueTests;
-
+
private: // but shared with ConcurrentQueue
std::uint32_t initialOffset;
std::uint32_t lastKnownGlobalOffset;
@@ -676,10 +769,10 @@ class ConcurrentQueue
public:
typedef ::moodycamel::ProducerToken producer_token_t;
typedef ::moodycamel::ConsumerToken consumer_token_t;
-
+
typedef typename Traits::index_t index_t;
typedef typename Traits::size_t size_t;
-
+
static const size_t BLOCK_SIZE = static_cast<size_t>(Traits::BLOCK_SIZE);
static const size_t EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD = static_cast<size_t>(Traits::EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD);
static const size_t EXPLICIT_INITIAL_INDEX_SIZE = static_cast<size_t>(Traits::EXPLICIT_INITIAL_INDEX_SIZE);
@@ -717,7 +810,7 @@ public:
// queue is fully constructed before it starts being used by other threads (this
// includes making the memory effects of construction visible, possibly with a
// memory barrier).
- explicit ConcurrentQueue(size_t capacity = 6 * BLOCK_SIZE)
+ explicit ConcurrentQueue(size_t capacity = 32 * BLOCK_SIZE)
: producerListTail(nullptr),
producerCount(0),
initialBlockPoolIndex(0),
@@ -727,7 +820,7 @@ public:
implicitProducerHashResizeInProgress.clear(std::memory_order_relaxed);
populate_initial_implicit_producer_hash();
populate_initial_block_list(capacity / BLOCK_SIZE + ((capacity & (BLOCK_SIZE - 1)) == 0 ? 0 : 1));
-
+
#ifdef MOODYCAMEL_QUEUE_INTERNAL_DEBUG
// Track all the producers using a fully-resolved typed list for
// each kind; this makes it possible to debug them starting from
@@ -737,7 +830,7 @@ public:
implicitProducers.store(nullptr, std::memory_order_relaxed);
#endif
}
-
+
// Computes the correct amount of pre-allocated blocks for you based
// on the minimum number of elements you want available at any given
// time, and the maximum concurrent number of each type of producer.
@@ -752,13 +845,13 @@ public:
populate_initial_implicit_producer_hash();
size_t blocks = (((minCapacity + BLOCK_SIZE - 1) / BLOCK_SIZE) - 1) * (maxExplicitProducers + 1) + 2 * (maxExplicitProducers + maxImplicitProducers);
populate_initial_block_list(blocks);
-
+
#ifdef MOODYCAMEL_QUEUE_INTERNAL_DEBUG
explicitProducers.store(nullptr, std::memory_order_relaxed);
implicitProducers.store(nullptr, std::memory_order_relaxed);
#endif
}
-
+
// Note: The queue should not be accessed concurrently while it's
// being deleted. It's up to the user to synchronize this.
// This method is not thread safe.
@@ -774,9 +867,9 @@ public:
destroy(ptr);
ptr = next;
}
-
+
// Destroy implicit producer hash tables
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE != 0) {
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE != 0) {
auto hash = implicitProducerHash.load(std::memory_order_relaxed);
while (hash != nullptr) {
auto prev = hash->prev;
@@ -790,7 +883,7 @@ public:
hash = prev;
}
}
-
+
// Destroy global free list
auto block = freeList.head_unsafe();
while (block != nullptr) {
@@ -800,7 +893,7 @@ public:
}
block = next;
}
-
+
// Destroy initial free list
destroy_array(initialBlockPool, initialBlockPoolSize);
}
@@ -808,7 +901,7 @@ public:
// Disable copying and copy assignment
ConcurrentQueue(ConcurrentQueue const&) MOODYCAMEL_DELETE_FUNCTION;
ConcurrentQueue& operator=(ConcurrentQueue const&) MOODYCAMEL_DELETE_FUNCTION;
-
+
// Moving is supported, but note that it is *not* a thread-safe operation.
// Nobody can use the queue while it's being moved, and the memory effects
// of that move must be propagated to other threads before they can use it.
@@ -829,31 +922,31 @@ public:
implicitProducerHashResizeInProgress.clear(std::memory_order_relaxed);
populate_initial_implicit_producer_hash();
swap_implicit_producer_hashes(other);
-
+
other.producerListTail.store(nullptr, std::memory_order_relaxed);
other.producerCount.store(0, std::memory_order_relaxed);
other.nextExplicitConsumerId.store(0, std::memory_order_relaxed);
other.globalExplicitConsumerOffset.store(0, std::memory_order_relaxed);
-
+
#ifdef MOODYCAMEL_QUEUE_INTERNAL_DEBUG
explicitProducers.store(other.explicitProducers.load(std::memory_order_relaxed), std::memory_order_relaxed);
other.explicitProducers.store(nullptr, std::memory_order_relaxed);
implicitProducers.store(other.implicitProducers.load(std::memory_order_relaxed), std::memory_order_relaxed);
other.implicitProducers.store(nullptr, std::memory_order_relaxed);
#endif
-
+
other.initialBlockPoolIndex.store(0, std::memory_order_relaxed);
other.initialBlockPoolSize = 0;
other.initialBlockPool = nullptr;
-
+
reown_producers();
}
-
+
inline ConcurrentQueue& operator=(ConcurrentQueue&& other) MOODYCAMEL_NOEXCEPT
{
return swap_internal(other);
}
-
+
// Swaps this queue's state with the other's. Not thread-safe.
// Swapping two queues does not invalidate their tokens, however
// the tokens that were created for one queue must be used with
@@ -863,14 +956,14 @@ public:
{
swap_internal(other);
}
-
+
private:
ConcurrentQueue& swap_internal(ConcurrentQueue& other)
{
if (this == &other) {
return *this;
}
-
+
details::swap_relaxed(producerListTail, other.producerListTail);
details::swap_relaxed(producerCount, other.producerCount);
details::swap_relaxed(initialBlockPoolIndex, other.initialBlockPoolIndex);
@@ -879,20 +972,20 @@ private:
freeList.swap(other.freeList);
details::swap_relaxed(nextExplicitConsumerId, other.nextExplicitConsumerId);
details::swap_relaxed(globalExplicitConsumerOffset, other.globalExplicitConsumerOffset);
-
+
swap_implicit_producer_hashes(other);
-
+
reown_producers();
other.reown_producers();
-
+
#ifdef MOODYCAMEL_QUEUE_INTERNAL_DEBUG
details::swap_relaxed(explicitProducers, other.explicitProducers);
details::swap_relaxed(implicitProducers, other.implicitProducers);
#endif
-
+
return *this;
}
-
+
public:
// Enqueues a single item (by copying it).
// Allocates memory if required. Only fails if memory allocation fails (or implicit
@@ -901,10 +994,10 @@ public:
// Thread-safe.
inline bool enqueue(T const& item)
{
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
- return inner_enqueue<CanAlloc>(item);
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
+ else return inner_enqueue<CanAlloc>(item);
}
-
+
// Enqueues a single item (by moving it, if possible).
// Allocates memory if required. Only fails if memory allocation fails (or implicit
// production is disabled because Traits::INITIAL_IMPLICIT_PRODUCER_HASH_SIZE is 0,
@@ -912,10 +1005,10 @@ public:
// Thread-safe.
inline bool enqueue(T&& item)
{
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
- return inner_enqueue<CanAlloc>(std::move(item));
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
+ else return inner_enqueue<CanAlloc>(std::move(item));
}
-
+
// Enqueues a single item (by copying it) using an explicit producer token.
// Allocates memory if required. Only fails if memory allocation fails (or
// Traits::MAX_SUBQUEUE_SIZE has been defined and would be surpassed).
@@ -924,7 +1017,7 @@ public:
{
return inner_enqueue<CanAlloc>(token, item);
}
-
+
// Enqueues a single item (by moving it, if possible) using an explicit producer token.
// Allocates memory if required. Only fails if memory allocation fails (or
// Traits::MAX_SUBQUEUE_SIZE has been defined and would be surpassed).
@@ -933,7 +1026,7 @@ public:
{
return inner_enqueue<CanAlloc>(token, std::move(item));
}
-
+
// Enqueues several items.
// Allocates memory if required. Only fails if memory allocation fails (or
// implicit production is disabled because Traits::INITIAL_IMPLICIT_PRODUCER_HASH_SIZE
@@ -943,10 +1036,10 @@ public:
template<typename It>
bool enqueue_bulk(It itemFirst, size_t count)
{
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
- return inner_enqueue_bulk<CanAlloc>(itemFirst, count);
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
+ else return inner_enqueue_bulk<CanAlloc>(itemFirst, count);
}
-
+
// Enqueues several items using an explicit producer token.
// Allocates memory if required. Only fails if memory allocation fails
// (or Traits::MAX_SUBQUEUE_SIZE has been defined and would be surpassed).
@@ -958,7 +1051,7 @@ public:
{
return inner_enqueue_bulk<CanAlloc>(token, itemFirst, count);
}
-
+
// Enqueues a single item (by copying it).
// Does not allocate memory. Fails if not enough room to enqueue (or implicit
// production is disabled because Traits::INITIAL_IMPLICIT_PRODUCER_HASH_SIZE
@@ -966,10 +1059,10 @@ public:
// Thread-safe.
inline bool try_enqueue(T const& item)
{
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
- return inner_enqueue<CannotAlloc>(item);
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
+ else return inner_enqueue<CannotAlloc>(item);
}
-
+
// Enqueues a single item (by moving it, if possible).
// Does not allocate memory (except for one-time implicit producer).
// Fails if not enough room to enqueue (or implicit production is
@@ -977,10 +1070,10 @@ public:
// Thread-safe.
inline bool try_enqueue(T&& item)
{
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
- return inner_enqueue<CannotAlloc>(std::move(item));
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
+ else return inner_enqueue<CannotAlloc>(std::move(item));
}
-
+
// Enqueues a single item (by copying it) using an explicit producer token.
// Does not allocate memory. Fails if not enough room to enqueue.
// Thread-safe.
@@ -988,7 +1081,7 @@ public:
{
return inner_enqueue<CannotAlloc>(token, item);
}
-
+
// Enqueues a single item (by moving it, if possible) using an explicit producer token.
// Does not allocate memory. Fails if not enough room to enqueue.
// Thread-safe.
@@ -996,7 +1089,7 @@ public:
{
return inner_enqueue<CannotAlloc>(token, std::move(item));
}
-
+
// Enqueues several items.
// Does not allocate memory (except for one-time implicit producer).
// Fails if not enough room to enqueue (or implicit production is
@@ -1007,10 +1100,10 @@ public:
template<typename It>
bool try_enqueue_bulk(It itemFirst, size_t count)
{
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
- return inner_enqueue_bulk<CannotAlloc>(itemFirst, count);
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return false;
+ else return inner_enqueue_bulk<CannotAlloc>(itemFirst, count);
}
-
+
// Enqueues several items using an explicit producer token.
// Does not allocate memory. Fails if not enough room to enqueue.
// Note: Use std::make_move_iterator if the elements should be moved
@@ -1021,9 +1114,9 @@ public:
{
return inner_enqueue_bulk<CannotAlloc>(token, itemFirst, count);
}
-
-
-
+
+
+
// Attempts to dequeue from the queue.
// Returns false if all producer streams appeared empty at the time they
// were checked (so, the queue is likely but not guaranteed to be empty).
@@ -1046,11 +1139,11 @@ public:
++nonEmptyCount;
}
}
-
+
// If there was at least one non-empty queue but it appears empty at the time
// we try to dequeue from it, we need to make sure every queue's been tried
if (nonEmptyCount > 0) {
- if (details::likely(best->dequeue(item))) {
+ if ((details::likely)(best->dequeue(item))) {
return true;
}
for (auto ptr = producerListTail.load(std::memory_order_acquire); ptr != nullptr; ptr = ptr->next_prod()) {
@@ -1061,7 +1154,7 @@ public:
}
return false;
}
-
+
// Attempts to dequeue from the queue.
// Returns false if all producer streams appeared empty at the time they
// were checked (so, the queue is likely but not guaranteed to be empty).
@@ -1081,7 +1174,7 @@ public:
}
return false;
}
-
+
// Attempts to dequeue from the queue using an explicit consumer token.
// Returns false if all producer streams appeared empty at the time they
// were checked (so, the queue is likely but not guaranteed to be empty).
@@ -1094,13 +1187,13 @@ public:
// If you see that the global offset has changed, you must reset your consumption counter and move to your designated place
// If there's no items where you're supposed to be, keep moving until you find a producer with some items
// If the global offset has not changed but you've run out of items to consume, move over from your current position until you find an producer with something in it
-
+
if (token.desiredProducer == nullptr || token.lastKnownGlobalOffset != globalExplicitConsumerOffset.load(std::memory_order_relaxed)) {
if (!update_current_producer_after_rotation(token)) {
return false;
}
}
-
+
// If there was at least one non-empty queue but it appears empty at the time
// we try to dequeue from it, we need to make sure every queue's been tried
if (static_cast<ProducerBase*>(token.currentProducer)->dequeue(item)) {
@@ -1109,7 +1202,7 @@ public:
}
return true;
}
-
+
auto tail = producerListTail.load(std::memory_order_acquire);
auto ptr = static_cast<ProducerBase*>(token.currentProducer)->next_prod();
if (ptr == nullptr) {
@@ -1128,7 +1221,7 @@ public:
}
return false;
}
-
+
// Attempts to dequeue several elements from the queue.
// Returns the number of items actually dequeued.
// Returns 0 if all producer streams appeared empty at the time they
@@ -1146,7 +1239,7 @@ public:
}
return count;
}
-
+
// Attempts to dequeue several elements from the queue using an explicit consumer token.
// Returns the number of items actually dequeued.
// Returns 0 if all producer streams appeared empty at the time they
@@ -1160,7 +1253,7 @@ public:
return 0;
}
}
-
+
size_t count = static_cast<ProducerBase*>(token.currentProducer)->dequeue_bulk(itemFirst, max);
if (count == max) {
if ((token.itemsConsumedFromCurrent += static_cast<std::uint32_t>(max)) >= EXPLICIT_CONSUMER_CONSUMPTION_QUOTA_BEFORE_ROTATE) {
@@ -1170,7 +1263,7 @@ public:
}
token.itemsConsumedFromCurrent += static_cast<std::uint32_t>(count);
max -= count;
-
+
auto tail = producerListTail.load(std::memory_order_acquire);
auto ptr = static_cast<ProducerBase*>(token.currentProducer)->next_prod();
if (ptr == nullptr) {
@@ -1194,9 +1287,9 @@ public:
}
return count;
}
-
-
-
+
+
+
// Attempts to dequeue from a specific producer's inner queue.
// If you happen to know which producer you want to dequeue from, this
// is significantly faster than using the general-case try_dequeue methods.
@@ -1208,7 +1301,7 @@ public:
{
return static_cast<ExplicitProducer*>(producer.producer)->dequeue(item);
}
-
+
// Attempts to dequeue several elements from a specific producer's inner queue.
// Returns the number of items actually dequeued.
// If you happen to know which producer you want to dequeue from, this
@@ -1221,8 +1314,8 @@ public:
{
return static_cast<ExplicitProducer*>(producer.producer)->dequeue_bulk(itemFirst, max);
}
-
-
+
+
// Returns an estimate of the total number of elements currently in the queue. This
// estimate is only accurate if the queue has completely stabilized before it is called
// (i.e. all enqueue and dequeue operations have completed and their memory effects are
@@ -1237,12 +1330,12 @@ public:
}
return size;
}
-
-
+
+
// Returns true if the underlying atomic variables used by
// the queue are lock-free (they should be on most platforms).
// Thread-safe.
- static bool is_lock_free()
+ static constexpr bool is_lock_free()
{
return
details::static_is_lock_free<bool>::value == 2 &&
@@ -1257,42 +1350,45 @@ public:
private:
friend struct ProducerToken;
friend struct ConsumerToken;
+ struct ExplicitProducer;
friend struct ExplicitProducer;
+ struct ImplicitProducer;
+ friend struct ImplicitProducer;
friend class ConcurrentQueueTests;
-
+
enum AllocationMode { CanAlloc, CannotAlloc };
-
-
+
+
///////////////////////////////
// Queue methods
///////////////////////////////
-
+
template<AllocationMode canAlloc, typename U>
inline bool inner_enqueue(producer_token_t const& token, U&& element)
{
return static_cast<ExplicitProducer*>(token.producer)->ConcurrentQueue::ExplicitProducer::template enqueue<canAlloc>(std::forward<U>(element));
}
-
+
template<AllocationMode canAlloc, typename U>
inline bool inner_enqueue(U&& element)
{
auto producer = get_or_add_implicit_producer();
return producer == nullptr ? false : producer->ConcurrentQueue::ImplicitProducer::template enqueue<canAlloc>(std::forward<U>(element));
}
-
+
template<AllocationMode canAlloc, typename It>
inline bool inner_enqueue_bulk(producer_token_t const& token, It itemFirst, size_t count)
{
return static_cast<ExplicitProducer*>(token.producer)->ConcurrentQueue::ExplicitProducer::template enqueue_bulk<canAlloc>(itemFirst, count);
}
-
+
template<AllocationMode canAlloc, typename It>
inline bool inner_enqueue_bulk(It itemFirst, size_t count)
{
auto producer = get_or_add_implicit_producer();
return producer == nullptr ? false : producer->ConcurrentQueue::ImplicitProducer::template enqueue_bulk<canAlloc>(itemFirst, count);
}
-
+
inline bool update_current_producer_after_rotation(consumer_token_t& token)
{
// Ah, there's been a rotation, figure out where we should be!
@@ -1302,7 +1398,7 @@ private:
}
auto prodCount = producerCount.load(std::memory_order_relaxed);
auto globalOffset = globalExplicitConsumerOffset.load(std::memory_order_relaxed);
- if (details::unlikely(token.desiredProducer == nullptr)) {
+ if ((details::unlikely)(token.desiredProducer == nullptr)) {
// Aha, first time we're dequeueing anything.
// Figure out our local position
// Note: offset is from start, not end, but we're traversing from end -- subtract from count first
@@ -1315,7 +1411,7 @@ private:
}
}
}
-
+
std::uint32_t delta = globalOffset - token.lastKnownGlobalOffset;
if (delta >= prodCount) {
delta = delta % prodCount;
@@ -1326,27 +1422,27 @@ private:
token.desiredProducer = tail;
}
}
-
+
token.lastKnownGlobalOffset = globalOffset;
token.currentProducer = token.desiredProducer;
token.itemsConsumedFromCurrent = 0;
return true;
}
-
-
+
+
///////////////////////////
// Free list
///////////////////////////
-
+
template <typename N>
struct FreeListNode
{
FreeListNode() : freeListRefs(0), freeListNext(nullptr) { }
-
+
std::atomic<std::uint32_t> freeListRefs;
std::atomic<N*> freeListNext;
};
-
+
// A simple CAS-based lock-free free list. Not the fastest thing in the world under heavy contention, but
// simple and correct (assuming nodes are never freed until after the free list is destroyed), and fairly
// speedy under low contention.
@@ -1356,15 +1452,15 @@ private:
FreeList() : freeListHead(nullptr) { }
FreeList(FreeList&& other) : freeListHead(other.freeListHead.load(std::memory_order_relaxed)) { other.freeListHead.store(nullptr, std::memory_order_relaxed); }
void swap(FreeList& other) { details::swap_relaxed(freeListHead, other.freeListHead); }
-
+
FreeList(FreeList const&) MOODYCAMEL_DELETE_FUNCTION;
FreeList& operator=(FreeList const&) MOODYCAMEL_DELETE_FUNCTION;
-
+
inline void add(N* node)
{
-#if MCDBGQ_NOLOCKFREE_FREELIST
+#ifdef MCDBGQ_NOLOCKFREE_FREELIST
debug::DebugLock lock(mutex);
-#endif
+#endif
// We know that the should-be-on-freelist bit is 0 at this point, so it's safe to
// set it using a fetch_add
if (node->freeListRefs.fetch_add(SHOULD_BE_ON_FREELIST, std::memory_order_acq_rel) == 0) {
@@ -1373,12 +1469,12 @@ private:
add_knowing_refcount_is_zero(node);
}
}
-
+
inline N* try_get()
{
-#if MCDBGQ_NOLOCKFREE_FREELIST
+#ifdef MCDBGQ_NOLOCKFREE_FREELIST
debug::DebugLock lock(mutex);
-#endif
+#endif
auto head = freeListHead.load(std::memory_order_acquire);
while (head != nullptr) {
auto prevHead = head;
@@ -1387,7 +1483,7 @@ private:
head = freeListHead.load(std::memory_order_acquire);
continue;
}
-
+
// Good, reference count has been incremented (it wasn't at zero), which means we can read the
// next and not worry about it changing between now and the time we do the CAS
auto next = head->freeListNext.load(std::memory_order_relaxed);
@@ -1395,27 +1491,27 @@ private:
// Yay, got the node. This means it was on the list, which means shouldBeOnFreeList must be false no
// matter the refcount (because nobody else knows it's been taken off yet, it can't have been put back on).
assert((head->freeListRefs.load(std::memory_order_relaxed) & SHOULD_BE_ON_FREELIST) == 0);
-
+
// Decrease refcount twice, once for our ref, and once for the list's ref
- head->freeListRefs.fetch_add(-2, std::memory_order_release);
+ head->freeListRefs.fetch_sub(2, std::memory_order_release);
return head;
}
-
+
// OK, the head must have changed on us, but we still need to decrease the refcount we increased.
// Note that we don't need to release any memory effects, but we do need to ensure that the reference
// count decrement happens-after the CAS on the head.
- refs = prevHead->freeListRefs.fetch_add(-1, std::memory_order_acq_rel);
+ refs = prevHead->freeListRefs.fetch_sub(1, std::memory_order_acq_rel);
if (refs == SHOULD_BE_ON_FREELIST + 1) {
add_knowing_refcount_is_zero(prevHead);
}
}
-
+
return nullptr;
}
-
+
// Useful for traversing the list when there's no contention (e.g. to destroy remaining nodes)
N* head_unsafe() const { return freeListHead.load(std::memory_order_relaxed); }
-
+
private:
inline void add_knowing_refcount_is_zero(N* node)
{
@@ -1440,47 +1536,47 @@ private:
return;
}
}
-
+
private:
// Implemented like a stack, but where node order doesn't matter (nodes are inserted out of order under contention)
std::atomic<N*> freeListHead;
-
+
static const std::uint32_t REFS_MASK = 0x7FFFFFFF;
static const std::uint32_t SHOULD_BE_ON_FREELIST = 0x80000000;
-
-#if MCDBGQ_NOLOCKFREE_FREELIST
+
+#ifdef MCDBGQ_NOLOCKFREE_FREELIST
debug::DebugMutex mutex;
#endif
};
-
-
+
+
///////////////////////////
// Block
///////////////////////////
-
+
enum InnerQueueContext { implicit_context = 0, explicit_context = 1 };
-
+
struct Block
{
Block()
- : next(nullptr), elementsCompletelyDequeued(0), freeListRefs(0), freeListNext(nullptr), shouldBeOnFreeList(false), dynamicallyAllocated(true)
+ : next(nullptr), elementsCompletelyDequeued(0), freeListRefs(0), freeListNext(nullptr), dynamicallyAllocated(true)
{
-#if MCDBGQ_TRACKMEM
+#ifdef MCDBGQ_TRACKMEM
owner = nullptr;
#endif
}
-
+
template<InnerQueueContext context>
inline bool is_empty() const
{
- if (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
+ MOODYCAMEL_CONSTEXPR_IF (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
// Check flags
for (size_t i = 0; i < BLOCK_SIZE; ++i) {
if (!emptyFlags[i].load(std::memory_order_relaxed)) {
return false;
}
}
-
+
// Aha, empty; make sure we have all other memory effects that happened before the empty flags were set
std::atomic_thread_fence(std::memory_order_acquire);
return true;
@@ -1495,12 +1591,12 @@ private:
return false;
}
}
-
+
// Returns true if the block is now empty (does not apply in explicit context)
template<InnerQueueContext context>
- inline bool set_empty(index_t i)
+ inline bool set_empty(MOODYCAMEL_MAYBE_UNUSED index_t i)
{
- if (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
+ MOODYCAMEL_CONSTEXPR_IF (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
// Set flag
assert(!emptyFlags[BLOCK_SIZE - 1 - static_cast<size_t>(i & static_cast<index_t>(BLOCK_SIZE - 1))].load(std::memory_order_relaxed));
emptyFlags[BLOCK_SIZE - 1 - static_cast<size_t>(i & static_cast<index_t>(BLOCK_SIZE - 1))].store(true, std::memory_order_release);
@@ -1513,13 +1609,13 @@ private:
return prevVal == BLOCK_SIZE - 1;
}
}
-
+
// Sets multiple contiguous item statuses to 'empty' (assumes no wrapping and count > 0).
// Returns true if the block is now empty (does not apply in explicit context).
template<InnerQueueContext context>
- inline bool set_many_empty(index_t i, size_t count)
+ inline bool set_many_empty(MOODYCAMEL_MAYBE_UNUSED index_t i, size_t count)
{
- if (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
+ MOODYCAMEL_CONSTEXPR_IF (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
// Set flags
std::atomic_thread_fence(std::memory_order_release);
i = BLOCK_SIZE - 1 - static_cast<size_t>(i & static_cast<index_t>(BLOCK_SIZE - 1)) - count + 1;
@@ -1536,11 +1632,11 @@ private:
return prevVal + count == BLOCK_SIZE;
}
}
-
+
template<InnerQueueContext context>
inline void set_all_empty()
{
- if (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
+ MOODYCAMEL_CONSTEXPR_IF (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
// Set all flags
for (size_t i = 0; i != BLOCK_SIZE; ++i) {
emptyFlags[i].store(true, std::memory_order_relaxed);
@@ -1551,11 +1647,11 @@ private:
elementsCompletelyDequeued.store(BLOCK_SIZE, std::memory_order_relaxed);
}
}
-
+
template<InnerQueueContext context>
inline void reset_empty()
{
- if (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
+ MOODYCAMEL_CONSTEXPR_IF (context == explicit_context && BLOCK_SIZE <= EXPLICIT_BLOCK_EMPTY_COUNTER_THRESHOLD) {
// Reset flags
for (size_t i = 0; i != BLOCK_SIZE; ++i) {
emptyFlags[i].store(false, std::memory_order_relaxed);
@@ -1566,25 +1662,13 @@ private:
elementsCompletelyDequeued.store(0, std::memory_order_relaxed);
}
}
-
+
inline T* operator[](index_t idx) MOODYCAMEL_NOEXCEPT { return static_cast<T*>(static_cast<void*>(elements)) + static_cast<size_t>(idx & static_cast<index_t>(BLOCK_SIZE - 1)); }
inline T const* operator[](index_t idx) const MOODYCAMEL_NOEXCEPT { return static_cast<T const*>(static_cast<void const*>(elements)) + static_cast<size_t>(idx & static_cast<index_t>(BLOCK_SIZE - 1)); }
-
+
private:
- // IMPORTANT: This must be the first member in Block, so that if T depends on the alignment of
- // addresses returned by malloc, that alignment will be preserved. Apparently clang actually
- // generates code that uses this assumption for AVX instructions in some cases. Ideally, we
- // should also align Block to the alignment of T in case it's higher than malloc's 16-byte
- // alignment, but this is hard to do in a cross-platform way. Assert for this case:
- static_assert(std::alignment_of<T>::value <= std::alignment_of<details::max_align_t>::value, "The queue does not support super-aligned types at this time");
- // Additionally, we need the alignment of Block itself to be a multiple of max_align_t since
- // otherwise the appropriate padding will not be added at the end of Block in order to make
- // arrays of Blocks all be properly aligned (not just the first one). We use a union to force
- // this.
- union {
- char elements[sizeof(T) * BLOCK_SIZE];
- details::max_align_t dummy;
- };
+ static_assert(std::alignment_of<T>::value <= sizeof(T), "The queue does not support types with an alignment greater than their size at this time");
+ MOODYCAMEL_ALIGNED_TYPE_LIKE(char[sizeof(T) * BLOCK_SIZE], T) elements;
public:
Block* next;
std::atomic<size_t> elementsCompletelyDequeued;
@@ -1592,26 +1676,25 @@ private:
public:
std::atomic<std::uint32_t> freeListRefs;
std::atomic<Block*> freeListNext;
- std::atomic<bool> shouldBeOnFreeList;
bool dynamicallyAllocated; // Perhaps a better name for this would be 'isNotPartOfInitialBlockPool'
-
-#if MCDBGQ_TRACKMEM
+
+#ifdef MCDBGQ_TRACKMEM
void* owner;
#endif
};
- static_assert(std::alignment_of<Block>::value >= std::alignment_of<details::max_align_t>::value, "Internal error: Blocks must be at least as aligned as the type they are wrapping");
+ static_assert(std::alignment_of<Block>::value >= std::alignment_of<T>::value, "Internal error: Blocks must be at least as aligned as the type they are wrapping");
-#if MCDBGQ_TRACKMEM
+#ifdef MCDBGQ_TRACKMEM
public:
struct MemStats;
private:
#endif
-
+
///////////////////////////
// Producer base
///////////////////////////
-
+
struct ProducerBase : public details::ConcurrentQueueProducerTypelessBase
{
ProducerBase(ConcurrentQueue* parent_, bool isExplicit_) :
@@ -1624,9 +1707,9 @@ private:
parent(parent_)
{
}
-
- virtual ~ProducerBase() { };
-
+
+ virtual ~ProducerBase() { }
+
template<typename U>
inline bool dequeue(U& element)
{
@@ -1637,7 +1720,7 @@ private:
return static_cast<ImplicitProducer*>(this)->dequeue(element);
}
}
-
+
template<typename It>
inline size_t dequeue_bulk(It& itemFirst, size_t max)
{
@@ -1648,45 +1731,45 @@ private:
return static_cast<ImplicitProducer*>(this)->dequeue_bulk(itemFirst, max);
}
}
-
+
inline ProducerBase* next_prod() const { return static_cast<ProducerBase*>(next); }
-
+
inline size_t size_approx() const
{
auto tail = tailIndex.load(std::memory_order_relaxed);
auto head = headIndex.load(std::memory_order_relaxed);
return details::circular_less_than(head, tail) ? static_cast<size_t>(tail - head) : 0;
}
-
+
inline index_t getTail() const { return tailIndex.load(std::memory_order_relaxed); }
protected:
std::atomic<index_t> tailIndex; // Where to enqueue to next
std::atomic<index_t> headIndex; // Where to dequeue from next
-
+
std::atomic<index_t> dequeueOptimisticCount;
std::atomic<index_t> dequeueOvercommit;
-
+
Block* tailBlock;
-
+
public:
bool isExplicit;
ConcurrentQueue* parent;
-
+
protected:
-#if MCDBGQ_TRACKMEM
+#ifdef MCDBGQ_TRACKMEM
friend struct MemStats;
#endif
};
-
-
+
+
///////////////////////////
// Explicit queue
///////////////////////////
-
+
struct ExplicitProducer : public ProducerBase
{
- explicit ExplicitProducer(ConcurrentQueue* parent) :
- ProducerBase(parent, true),
+ explicit ExplicitProducer(ConcurrentQueue* parent_) :
+ ProducerBase(parent_, true),
blockIndex(nullptr),
pr_blockIndexSlotsUsed(0),
pr_blockIndexSize(EXPLICIT_INITIAL_INDEX_SIZE >> 1),
@@ -1694,14 +1777,14 @@ private:
pr_blockIndexEntries(nullptr),
pr_blockIndexRaw(nullptr)
{
- size_t poolBasedIndexSize = details::ceil_to_pow_2(parent->initialBlockPoolSize) >> 1;
+ size_t poolBasedIndexSize = details::ceil_to_pow_2(parent_->initialBlockPoolSize) >> 1;
if (poolBasedIndexSize > pr_blockIndexSize) {
pr_blockIndexSize = poolBasedIndexSize;
}
-
+
new_block_index(0); // This creates an index with double the number of current entries, i.e. EXPLICIT_INITIAL_INDEX_SIZE
}
-
+
~ExplicitProducer()
{
// Destruct any elements not yet dequeued.
@@ -1720,7 +1803,7 @@ private:
assert(details::circular_less_than<index_t>(pr_blockIndexEntries[i].base, this->headIndex.load(std::memory_order_relaxed)));
halfDequeuedBlock = pr_blockIndexEntries[i].block;
}
-
+
// Start at the head block (note the first line in the loop gives us the head from the tail on the first iteration)
auto block = this->tailBlock;
do {
@@ -1728,12 +1811,12 @@ private:
if (block->ConcurrentQueue::Block::template is_empty<explicit_context>()) {
continue;
}
-
+
size_t i = 0; // Offset into block
if (block == halfDequeuedBlock) {
i = static_cast<size_t>(this->headIndex.load(std::memory_order_relaxed) & static_cast<index_t>(BLOCK_SIZE - 1));
}
-
+
// Walk through all the items in the block; if this is the tail block, we need to stop when we reach the tail index
auto lastValidIndex = (this->tailIndex.load(std::memory_order_relaxed) & static_cast<index_t>(BLOCK_SIZE - 1)) == 0 ? BLOCK_SIZE : static_cast<size_t>(this->tailIndex.load(std::memory_order_relaxed) & static_cast<index_t>(BLOCK_SIZE - 1));
while (i != BLOCK_SIZE && (block != this->tailBlock || i != lastValidIndex)) {
@@ -1741,22 +1824,17 @@ private:
}
} while (block != this->tailBlock);
}
-
+
// Destroy all blocks that we own
if (this->tailBlock != nullptr) {
auto block = this->tailBlock;
do {
auto nextBlock = block->next;
- if (block->dynamicallyAllocated) {
- destroy(block);
- }
- else {
- this->parent->add_block_to_free_list(block);
- }
+ this->parent->add_block_to_free_list(block);
block = nextBlock;
} while (block != this->tailBlock);
}
-
+
// Destroy the block indices
auto header = static_cast<BlockIndexHeader*>(pr_blockIndexRaw);
while (header != nullptr) {
@@ -1766,7 +1844,7 @@ private:
header = prev;
}
}
-
+
template<AllocationMode allocMode, typename U>
inline bool enqueue(U&& element)
{
@@ -1777,10 +1855,10 @@ private:
auto startBlock = this->tailBlock;
auto originalBlockIndexSlotsUsed = pr_blockIndexSlotsUsed;
if (this->tailBlock != nullptr && this->tailBlock->next->ConcurrentQueue::Block::template is_empty<explicit_context>()) {
- // We can re-use the block ahead of us, it's empty!
+ // We can re-use the block ahead of us, it's empty!
this->tailBlock = this->tailBlock->next;
this->tailBlock->ConcurrentQueue::Block::template reset_empty<explicit_context>();
-
+
// We'll put the block on the block index (guaranteed to be room since we're conceptually removing the
// last block from it first -- except instead of removing then adding, we can just overwrite).
// Note that there must be a valid block index here, since even if allocation failed in the ctor,
@@ -1805,18 +1883,21 @@ private:
// Hmm, the circular block index is already full -- we'll need
// to allocate a new index. Note pr_blockIndexRaw can only be nullptr if
// the initial allocation failed in the constructor.
-
- if (allocMode == CannotAlloc || !new_block_index(pr_blockIndexSlotsUsed)) {
+
+ MOODYCAMEL_CONSTEXPR_IF (allocMode == CannotAlloc) {
+ return false;
+ }
+ else if (!new_block_index(pr_blockIndexSlotsUsed)) {
return false;
}
}
-
+
// Insert a new block in the circular linked list
auto newBlock = this->parent->ConcurrentQueue::template requisition_block<allocMode>();
if (newBlock == nullptr) {
return false;
}
-#if MCDBGQ_TRACKMEM
+#ifdef MCDBGQ_TRACKMEM
newBlock->owner = this;
#endif
newBlock->ConcurrentQueue::Block::template reset_empty<explicit_context>();
@@ -1830,8 +1911,8 @@ private:
this->tailBlock = newBlock;
++pr_blockIndexSlotsUsed;
}
-
- if (!MOODYCAMEL_NOEXCEPT_CTOR(T, U, new (nullptr) T(std::forward<U>(element)))) {
+
+ MOODYCAMEL_CONSTEXPR_IF (!MOODYCAMEL_NOEXCEPT_CTOR(T, U, new (static_cast<T*>(nullptr)) T(std::forward<U>(element)))) {
// The constructor may throw. We want the element not to appear in the queue in
// that case (without corrupting the queue):
MOODYCAMEL_TRY {
@@ -1849,27 +1930,27 @@ private:
(void)startBlock;
(void)originalBlockIndexSlotsUsed;
}
-
+
// Add block to block index
auto& entry = blockIndex.load(std::memory_order_relaxed)->entries[pr_blockIndexFront];
entry.base = currentTailIndex;
entry.block = this->tailBlock;
blockIndex.load(std::memory_order_relaxed)->front.store(pr_blockIndexFront, std::memory_order_release);
pr_blockIndexFront = (pr_blockIndexFront + 1) & (pr_blockIndexSize - 1);
-
- if (!MOODYCAMEL_NOEXCEPT_CTOR(T, U, new (nullptr) T(std::forward<U>(element)))) {
+
+ MOODYCAMEL_CONSTEXPR_IF (!MOODYCAMEL_NOEXCEPT_CTOR(T, U, new (static_cast<T*>(nullptr)) T(std::forward<U>(element)))) {
this->tailIndex.store(newTailIndex, std::memory_order_release);
return true;
}
}
-
+
// Enqueue
new ((*this->tailBlock)[currentTailIndex]) T(std::forward<U>(element));
-
+
this->tailIndex.store(newTailIndex, std::memory_order_release);
return true;
}
-
+
template<typename U>
bool dequeue(U& element)
{
@@ -1877,10 +1958,10 @@ private:
auto overcommit = this->dequeueOvercommit.load(std::memory_order_relaxed);
if (details::circular_less_than<index_t>(this->dequeueOptimisticCount.load(std::memory_order_relaxed) - overcommit, tail)) {
// Might be something to dequeue, let's give it a try
-
+
// Note that this if is purely for performance purposes in the common case when the queue is
// empty and the values are eventually consistent -- we may enter here spuriously.
-
+
// Note that whatever the values of overcommit and tail are, they are not going to change (unless we
// change them) and must be the same value at this point (inside the if) as when the if condition was
// evaluated.
@@ -1893,23 +1974,24 @@ private:
// unfortunately that can't be shown to be correct using only the C++11 standard.
// See http://stackoverflow.com/questions/18223161/what-are-the-c11-memory-ordering-guarantees-in-this-corner-case
std::atomic_thread_fence(std::memory_order_acquire);
-
+
// Increment optimistic counter, then check if it went over the boundary
auto myDequeueCount = this->dequeueOptimisticCount.fetch_add(1, std::memory_order_relaxed);
-
+
// Note that since dequeueOvercommit must be <= dequeueOptimisticCount (because dequeueOvercommit is only ever
// incremented after dequeueOptimisticCount -- this is enforced in the `else` block below), and since we now
// have a version of dequeueOptimisticCount that is at least as recent as overcommit (due to the release upon
// incrementing dequeueOvercommit and the acquire above that synchronizes with it), overcommit <= myDequeueCount.
- assert(overcommit <= myDequeueCount);
-
+ // However, we can't assert this since both dequeueOptimisticCount and dequeueOvercommit may (independently)
+ // overflow; in such a case, though, the logic still holds since the difference between the two is maintained.
+
// Note that we reload tail here in case it changed; it will be the same value as before or greater, since
// this load is sequenced after (happens after) the earlier load above. This is supported by read-read
// coherency (as defined in the standard), explained here: http://en.cppreference.com/w/cpp/atomic/memory_order
tail = this->tailIndex.load(std::memory_order_acquire);
- if (details::likely(details::circular_less_than<index_t>(myDequeueCount - overcommit, tail))) {
+ if ((details::likely)(details::circular_less_than<index_t>(myDequeueCount - overcommit, tail))) {
// Guaranteed to be at least one element to dequeue!
-
+
// Get the index. Note that since there's guaranteed to be at least one element, this
// will never exceed tail. We need to do an acquire-release fence here since it's possible
// that whatever condition got us to this point was for an earlier enqueued element (that
@@ -1919,21 +2001,21 @@ private:
// place with the more current condition (they must have acquired a tail that is at least
// as recent).
auto index = this->headIndex.fetch_add(1, std::memory_order_acq_rel);
-
-
+
+
// Determine which block the element is in
-
+
auto localBlockIndex = blockIndex.load(std::memory_order_acquire);
auto localBlockIndexHead = localBlockIndex->front.load(std::memory_order_acquire);
-
+
// We need to be careful here about subtracting and dividing because of index wrap-around.
// When an index wraps, we need to preserve the sign of the offset when dividing it by the
// block size (in order to get a correct signed block count offset in all cases):
auto headBase = localBlockIndex->entries[localBlockIndexHead].base;
auto blockBaseIndex = index & ~static_cast<index_t>(BLOCK_SIZE - 1);
- auto offset = static_cast<size_t>(static_cast<typename std::make_signed<index_t>::type>(blockBaseIndex - headBase) / BLOCK_SIZE);
+ auto offset = static_cast<size_t>(static_cast<typename std::make_signed<index_t>::type>(blockBaseIndex - headBase) / static_cast<typename std::make_signed<index_t>::type>(BLOCK_SIZE));
auto block = localBlockIndex->entries[(localBlockIndexHead + offset) & (localBlockIndex->size - 1)].block;
-
+
// Dequeue
auto& el = *((*block)[index]);
if (!MOODYCAMEL_NOEXCEPT_ASSIGN(T, T&&, element = std::move(el))) {
@@ -1942,22 +2024,22 @@ private:
struct Guard {
Block* block;
index_t index;
-
+
~Guard()
{
(*block)[index]->~T();
block->ConcurrentQueue::Block::template set_empty<explicit_context>(index);
}
} guard = { block, index };
-
- element = std::move(el);
+
+ element = std::move(el); // NOLINT
}
else {
- element = std::move(el);
- el.~T();
+ element = std::move(el); // NOLINT
+ el.~T(); // NOLINT
block->ConcurrentQueue::Block::template set_empty<explicit_context>(index);
}
-
+
return true;
}
else {
@@ -1965,12 +2047,12 @@ private:
this->dequeueOvercommit.fetch_add(1, std::memory_order_release); // Release so that the fetch_add on dequeueOptimisticCount is guaranteed to happen before this write
}
}
-
+
return false;
}
-
+
template<AllocationMode allocMode, typename It>
- bool enqueue_bulk(It itemFirst, size_t count)
+ bool MOODYCAMEL_NO_TSAN enqueue_bulk(It itemFirst, size_t count)
{
// First, we need to make sure we have enough room to enqueue all of the elements;
// this means pre-allocating blocks and putting them in the block index (but only if
@@ -1979,9 +2061,9 @@ private:
auto startBlock = this->tailBlock;
auto originalBlockIndexFront = pr_blockIndexFront;
auto originalBlockIndexSlotsUsed = pr_blockIndexSlotsUsed;
-
+
Block* firstAllocatedBlock = nullptr;
-
+
// Figure out how many blocks we'll need to allocate, and do so
size_t blockBaseDiff = ((startTailIndex + count - 1) & ~static_cast<index_t>(BLOCK_SIZE - 1)) - ((startTailIndex - 1) & ~static_cast<index_t>(BLOCK_SIZE - 1));
index_t currentTailIndex = (startTailIndex - 1) & ~static_cast<index_t>(BLOCK_SIZE - 1);
@@ -1990,39 +2072,46 @@ private:
while (blockBaseDiff > 0 && this->tailBlock != nullptr && this->tailBlock->next != firstAllocatedBlock && this->tailBlock->next->ConcurrentQueue::Block::template is_empty<explicit_context>()) {
blockBaseDiff -= static_cast<index_t>(BLOCK_SIZE);
currentTailIndex += static_cast<index_t>(BLOCK_SIZE);
-
+
this->tailBlock = this->tailBlock->next;
firstAllocatedBlock = firstAllocatedBlock == nullptr ? this->tailBlock : firstAllocatedBlock;
-
+
auto& entry = blockIndex.load(std::memory_order_relaxed)->entries[pr_blockIndexFront];
entry.base = currentTailIndex;
entry.block = this->tailBlock;
pr_blockIndexFront = (pr_blockIndexFront + 1) & (pr_blockIndexSize - 1);
}
-
+
// Now allocate as many blocks as necessary from the block pool
while (blockBaseDiff > 0) {
blockBaseDiff -= static_cast<index_t>(BLOCK_SIZE);
currentTailIndex += static_cast<index_t>(BLOCK_SIZE);
-
+
auto head = this->headIndex.load(std::memory_order_relaxed);
assert(!details::circular_less_than<index_t>(currentTailIndex, head));
bool full = !details::circular_less_than<index_t>(head, currentTailIndex + BLOCK_SIZE) || (MAX_SUBQUEUE_SIZE != details::const_numeric_max<size_t>::value && (MAX_SUBQUEUE_SIZE == 0 || MAX_SUBQUEUE_SIZE - BLOCK_SIZE < currentTailIndex - head));
if (pr_blockIndexRaw == nullptr || pr_blockIndexSlotsUsed == pr_blockIndexSize || full) {
- if (allocMode == CannotAlloc || full || !new_block_index(originalBlockIndexSlotsUsed)) {
+ MOODYCAMEL_CONSTEXPR_IF (allocMode == CannotAlloc) {
// Failed to allocate, undo changes (but keep injected blocks)
pr_blockIndexFront = originalBlockIndexFront;
pr_blockIndexSlotsUsed = originalBlockIndexSlotsUsed;
this->tailBlock = startBlock == nullptr ? firstAllocatedBlock : startBlock;
return false;
}
-
+ else if (full || !new_block_index(originalBlockIndexSlotsUsed)) {
+ // Failed to allocate, undo changes (but keep injected blocks)
+ pr_blockIndexFront = originalBlockIndexFront;
+ pr_blockIndexSlotsUsed = originalBlockIndexSlotsUsed;
+ this->tailBlock = startBlock == nullptr ? firstAllocatedBlock : startBlock;
+ return false;
+ }
+
// pr_blockIndexFront is updated inside new_block_index, so we need to
// update our fallback value too (since we keep the new index even if we
// later fail)
originalBlockIndexFront = originalBlockIndexSlotsUsed;
}
-
+
// Insert a new block in the circular linked list
auto newBlock = this->parent->ConcurrentQueue::template requisition_block<allocMode>();
if (newBlock == nullptr) {
@@ -2031,8 +2120,8 @@ private:
this->tailBlock = startBlock == nullptr ? firstAllocatedBlock : startBlock;
return false;
}
-
-#if MCDBGQ_TRACKMEM
+
+#ifdef MCDBGQ_TRACKMEM
newBlock->owner = this;
#endif
newBlock->ConcurrentQueue::Block::template set_all_empty<explicit_context>();
@@ -2045,15 +2134,15 @@ private:
}
this->tailBlock = newBlock;
firstAllocatedBlock = firstAllocatedBlock == nullptr ? this->tailBlock : firstAllocatedBlock;
-
+
++pr_blockIndexSlotsUsed;
-
+
auto& entry = blockIndex.load(std::memory_order_relaxed)->entries[pr_blockIndexFront];
entry.base = currentTailIndex;
entry.block = this->tailBlock;
pr_blockIndexFront = (pr_blockIndexFront + 1) & (pr_blockIndexSize - 1);
}
-
+
// Excellent, all allocations succeeded. Reset each block's emptiness before we fill them up, and
// publish the new block index front
auto block = firstAllocatedBlock;
@@ -2064,12 +2153,12 @@ private:
}
block = block->next;
}
-
- if (MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (nullptr) T(details::deref_noexcept(itemFirst)))) {
+
+ MOODYCAMEL_CONSTEXPR_IF (MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (static_cast<T*>(nullptr)) T(details::deref_noexcept(itemFirst)))) {
blockIndex.load(std::memory_order_relaxed)->front.store((pr_blockIndexFront - 1) & (pr_blockIndexSize - 1), std::memory_order_release);
}
}
-
+
// Enqueue, one block at a time
index_t newTailIndex = startTailIndex + static_cast<index_t>(count);
currentTailIndex = startTailIndex;
@@ -2080,11 +2169,11 @@ private:
this->tailBlock = firstAllocatedBlock;
}
while (true) {
- auto stopIndex = (currentTailIndex & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
+ index_t stopIndex = (currentTailIndex & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
if (details::circular_less_than<index_t>(newTailIndex, stopIndex)) {
stopIndex = newTailIndex;
}
- if (MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (nullptr) T(details::deref_noexcept(itemFirst)))) {
+ MOODYCAMEL_CONSTEXPR_IF (MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (static_cast<T*>(nullptr)) T(details::deref_noexcept(itemFirst)))) {
while (currentTailIndex != stopIndex) {
new ((*this->tailBlock)[currentTailIndex++]) T(*itemFirst++);
}
@@ -2099,7 +2188,7 @@ private:
// may only define a (noexcept) move constructor, and so calls to the
// cctor will not compile, even if they are in an if branch that will never
// be executed
- new ((*this->tailBlock)[currentTailIndex]) T(details::nomove_if<(bool)!MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (nullptr) T(details::deref_noexcept(itemFirst)))>::eval(*itemFirst));
+ new ((*this->tailBlock)[currentTailIndex]) T(details::nomove_if<!MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (static_cast<T*>(nullptr)) T(details::deref_noexcept(itemFirst)))>::eval(*itemFirst));
++currentTailIndex;
++itemFirst;
}
@@ -2110,11 +2199,11 @@ private:
// any allocated blocks in our linked list for later, though).
auto constructedStopIndex = currentTailIndex;
auto lastBlockEnqueued = this->tailBlock;
-
+
pr_blockIndexFront = originalBlockIndexFront;
pr_blockIndexSlotsUsed = originalBlockIndexSlotsUsed;
this->tailBlock = startBlock == nullptr ? firstAllocatedBlock : startBlock;
-
+
if (!details::is_trivially_destructible<T>::value) {
auto block = startBlock;
if ((startTailIndex & static_cast<index_t>(BLOCK_SIZE - 1)) == 0) {
@@ -2138,22 +2227,23 @@ private:
MOODYCAMEL_RETHROW;
}
}
-
+
if (this->tailBlock == endBlock) {
assert(currentTailIndex == newTailIndex);
break;
}
this->tailBlock = this->tailBlock->next;
}
-
- if (!MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (nullptr) T(details::deref_noexcept(itemFirst))) && firstAllocatedBlock != nullptr) {
- blockIndex.load(std::memory_order_relaxed)->front.store((pr_blockIndexFront - 1) & (pr_blockIndexSize - 1), std::memory_order_release);
+
+ MOODYCAMEL_CONSTEXPR_IF (!MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (static_cast<T*>(nullptr)) T(details::deref_noexcept(itemFirst)))) {
+ if (firstAllocatedBlock != nullptr)
+ blockIndex.load(std::memory_order_relaxed)->front.store((pr_blockIndexFront - 1) & (pr_blockIndexSize - 1), std::memory_order_release);
}
-
+
this->tailIndex.store(newTailIndex, std::memory_order_release);
return true;
}
-
+
template<typename It>
size_t dequeue_bulk(It& itemFirst, size_t max)
{
@@ -2163,10 +2253,9 @@ private:
if (details::circular_less_than<size_t>(0, desiredCount)) {
desiredCount = desiredCount < max ? desiredCount : max;
std::atomic_thread_fence(std::memory_order_acquire);
-
+
auto myDequeueCount = this->dequeueOptimisticCount.fetch_add(desiredCount, std::memory_order_relaxed);
- assert(overcommit <= myDequeueCount);
-
+
tail = this->tailIndex.load(std::memory_order_acquire);
auto actualCount = static_cast<size_t>(tail - (myDequeueCount - overcommit));
if (details::circular_less_than<size_t>(0, actualCount)) {
@@ -2174,25 +2263,25 @@ private:
if (actualCount < desiredCount) {
this->dequeueOvercommit.fetch_add(desiredCount - actualCount, std::memory_order_release);
}
-
+
// Get the first index. Note that since there's guaranteed to be at least actualCount elements, this
// will never exceed tail.
auto firstIndex = this->headIndex.fetch_add(actualCount, std::memory_order_acq_rel);
-
+
// Determine which block the first element is in
auto localBlockIndex = blockIndex.load(std::memory_order_acquire);
auto localBlockIndexHead = localBlockIndex->front.load(std::memory_order_acquire);
-
+
auto headBase = localBlockIndex->entries[localBlockIndexHead].base;
auto firstBlockBaseIndex = firstIndex & ~static_cast<index_t>(BLOCK_SIZE - 1);
- auto offset = static_cast<size_t>(static_cast<typename std::make_signed<index_t>::type>(firstBlockBaseIndex - headBase) / BLOCK_SIZE);
+ auto offset = static_cast<size_t>(static_cast<typename std::make_signed<index_t>::type>(firstBlockBaseIndex - headBase) / static_cast<typename std::make_signed<index_t>::type>(BLOCK_SIZE));
auto indexIndex = (localBlockIndexHead + offset) & (localBlockIndex->size - 1);
-
+
// Iterate the blocks and dequeue
auto index = firstIndex;
do {
auto firstIndexInBlock = index;
- auto endIndex = (index & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
+ index_t endIndex = (index & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
endIndex = details::circular_less_than<index_t>(firstIndex + static_cast<index_t>(actualCount), endIndex) ? firstIndex + static_cast<index_t>(actualCount) : endIndex;
auto block = localBlockIndex->entries[indexIndex].block;
if (MOODYCAMEL_NOEXCEPT_ASSIGN(T, T&&, details::deref_noexcept(itemFirst) = std::move((*(*block)[index])))) {
@@ -2224,19 +2313,19 @@ private:
}
block->ConcurrentQueue::Block::template set_many_empty<explicit_context>(firstIndexInBlock, static_cast<size_t>(endIndex - firstIndexInBlock));
indexIndex = (indexIndex + 1) & (localBlockIndex->size - 1);
-
+
firstIndexInBlock = index;
endIndex = (index & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
endIndex = details::circular_less_than<index_t>(firstIndex + static_cast<index_t>(actualCount), endIndex) ? firstIndex + static_cast<index_t>(actualCount) : endIndex;
} while (index != firstIndex + actualCount);
-
+
MOODYCAMEL_RETHROW;
}
}
block->ConcurrentQueue::Block::template set_many_empty<explicit_context>(firstIndexInBlock, static_cast<size_t>(endIndex - firstIndexInBlock));
indexIndex = (indexIndex + 1) & (localBlockIndex->size - 1);
} while (index != firstIndex + actualCount);
-
+
return actualCount;
}
else {
@@ -2244,17 +2333,17 @@ private:
this->dequeueOvercommit.fetch_add(desiredCount, std::memory_order_release);
}
}
-
+
return 0;
}
-
+
private:
struct BlockIndexEntry
{
index_t base;
Block* block;
};
-
+
struct BlockIndexHeader
{
size_t size;
@@ -2262,12 +2351,12 @@ private:
BlockIndexEntry* entries;
void* prev;
};
-
-
+
+
bool new_block_index(size_t numberOfFilledSlotsToExpose)
{
auto prevBlockSizeMask = pr_blockIndexSize - 1;
-
+
// Create the new block
pr_blockIndexSize <<= 1;
auto newRawPtr = static_cast<char*>((Traits::malloc)(sizeof(BlockIndexHeader) + std::alignment_of<BlockIndexEntry>::value - 1 + sizeof(BlockIndexEntry) * pr_blockIndexSize));
@@ -2275,9 +2364,9 @@ private:
pr_blockIndexSize >>= 1; // Reset to allow graceful retry
return false;
}
-
+
auto newBlockIndexEntries = reinterpret_cast<BlockIndexEntry*>(details::align_for<BlockIndexEntry>(newRawPtr + sizeof(BlockIndexHeader)));
-
+
// Copy in all the old indices, if any
size_t j = 0;
if (pr_blockIndexSlotsUsed != 0) {
@@ -2287,72 +2376,72 @@ private:
i = (i + 1) & prevBlockSizeMask;
} while (i != pr_blockIndexFront);
}
-
+
// Update everything
auto header = new (newRawPtr) BlockIndexHeader;
header->size = pr_blockIndexSize;
header->front.store(numberOfFilledSlotsToExpose - 1, std::memory_order_relaxed);
header->entries = newBlockIndexEntries;
header->prev = pr_blockIndexRaw; // we link the new block to the old one so we can free it later
-
+
pr_blockIndexFront = j;
pr_blockIndexEntries = newBlockIndexEntries;
pr_blockIndexRaw = newRawPtr;
blockIndex.store(header, std::memory_order_release);
-
+
return true;
}
-
+
private:
std::atomic<BlockIndexHeader*> blockIndex;
-
+
// To be used by producer only -- consumer must use the ones in referenced by blockIndex
size_t pr_blockIndexSlotsUsed;
size_t pr_blockIndexSize;
size_t pr_blockIndexFront; // Next slot (not current)
BlockIndexEntry* pr_blockIndexEntries;
void* pr_blockIndexRaw;
-
+
#ifdef MOODYCAMEL_QUEUE_INTERNAL_DEBUG
public:
ExplicitProducer* nextExplicitProducer;
private:
#endif
-
-#if MCDBGQ_TRACKMEM
+
+#ifdef MCDBGQ_TRACKMEM
friend struct MemStats;
#endif
};
-
-
+
+
//////////////////////////////////
// Implicit queue
//////////////////////////////////
-
+
struct ImplicitProducer : public ProducerBase
- {
- ImplicitProducer(ConcurrentQueue* parent) :
- ProducerBase(parent, false),
+ {
+ ImplicitProducer(ConcurrentQueue* parent_) :
+ ProducerBase(parent_, false),
nextBlockIndexCapacity(IMPLICIT_INITIAL_INDEX_SIZE),
blockIndex(nullptr)
{
new_block_index();
}
-
+
~ImplicitProducer()
{
// Note that since we're in the destructor we can assume that all enqueue/dequeue operations
// completed already; this means that all undequeued elements are placed contiguously across
// contiguous blocks, and that only the first and last remaining blocks can be only partially
// empty (all other remaining blocks must be completely full).
-
+
#ifdef MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED
// Unregister ourselves for thread termination notification
if (!this->inactive.load(std::memory_order_relaxed)) {
details::ThreadExitNotifier::unsubscribe(&threadExitListener);
}
#endif
-
+
// Destroy all remaining elements!
auto tail = this->tailIndex.load(std::memory_order_relaxed);
auto index = this->headIndex.load(std::memory_order_relaxed);
@@ -2365,10 +2454,10 @@ private:
// Free the old block
this->parent->add_block_to_free_list(block);
}
-
+
block = get_block_index_entry_for_index(index)->value.load(std::memory_order_relaxed);
}
-
+
((*block)[index])->~T();
++index;
}
@@ -2378,7 +2467,7 @@ private:
if (this->tailBlock != nullptr && (forceFreeLastBlock || (tail & static_cast<index_t>(BLOCK_SIZE - 1)) != 0)) {
this->parent->add_block_to_free_list(this->tailBlock);
}
-
+
// Destroy block index
auto localBlockIndex = blockIndex.load(std::memory_order_relaxed);
if (localBlockIndex != nullptr) {
@@ -2393,7 +2482,7 @@ private:
} while (localBlockIndex != nullptr);
}
}
-
+
template<AllocationMode allocMode, typename U>
inline bool enqueue(U&& element)
{
@@ -2406,7 +2495,7 @@ private:
if (!details::circular_less_than<index_t>(head, currentTailIndex + BLOCK_SIZE) || (MAX_SUBQUEUE_SIZE != details::const_numeric_max<size_t>::value && (MAX_SUBQUEUE_SIZE == 0 || MAX_SUBQUEUE_SIZE - BLOCK_SIZE < currentTailIndex - head))) {
return false;
}
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
debug::DebugLock lock(mutex);
#endif
// Find out where we'll be inserting this block in the block index
@@ -2414,7 +2503,7 @@ private:
if (!insert_block_index_entry<allocMode>(idxEntry, currentTailIndex)) {
return false;
}
-
+
// Get ahold of a new block
auto newBlock = this->parent->ConcurrentQueue::template requisition_block<allocMode>();
if (newBlock == nullptr) {
@@ -2422,12 +2511,12 @@ private:
idxEntry->value.store(nullptr, std::memory_order_relaxed);
return false;
}
-#if MCDBGQ_TRACKMEM
+#ifdef MCDBGQ_TRACKMEM
newBlock->owner = this;
#endif
newBlock->ConcurrentQueue::Block::template reset_empty<implicit_context>();
-
- if (!MOODYCAMEL_NOEXCEPT_CTOR(T, U, new (nullptr) T(std::forward<U>(element)))) {
+
+ MOODYCAMEL_CONSTEXPR_IF (!MOODYCAMEL_NOEXCEPT_CTOR(T, U, new (static_cast<T*>(nullptr)) T(std::forward<U>(element)))) {
// May throw, try to insert now before we publish the fact that we have this new block
MOODYCAMEL_TRY {
new ((*newBlock)[currentTailIndex]) T(std::forward<U>(element));
@@ -2439,25 +2528,25 @@ private:
MOODYCAMEL_RETHROW;
}
}
-
+
// Insert the new block into the index
idxEntry->value.store(newBlock, std::memory_order_relaxed);
-
+
this->tailBlock = newBlock;
-
- if (!MOODYCAMEL_NOEXCEPT_CTOR(T, U, new (nullptr) T(std::forward<U>(element)))) {
+
+ MOODYCAMEL_CONSTEXPR_IF (!MOODYCAMEL_NOEXCEPT_CTOR(T, U, new (static_cast<T*>(nullptr)) T(std::forward<U>(element)))) {
this->tailIndex.store(newTailIndex, std::memory_order_release);
return true;
}
}
-
+
// Enqueue
new ((*this->tailBlock)[currentTailIndex]) T(std::forward<U>(element));
-
+
this->tailIndex.store(newTailIndex, std::memory_order_release);
return true;
}
-
+
template<typename U>
bool dequeue(U& element)
{
@@ -2466,22 +2555,21 @@ private:
index_t overcommit = this->dequeueOvercommit.load(std::memory_order_relaxed);
if (details::circular_less_than<index_t>(this->dequeueOptimisticCount.load(std::memory_order_relaxed) - overcommit, tail)) {
std::atomic_thread_fence(std::memory_order_acquire);
-
+
index_t myDequeueCount = this->dequeueOptimisticCount.fetch_add(1, std::memory_order_relaxed);
- assert(overcommit <= myDequeueCount);
tail = this->tailIndex.load(std::memory_order_acquire);
- if (details::likely(details::circular_less_than<index_t>(myDequeueCount - overcommit, tail))) {
+ if ((details::likely)(details::circular_less_than<index_t>(myDequeueCount - overcommit, tail))) {
index_t index = this->headIndex.fetch_add(1, std::memory_order_acq_rel);
-
+
// Determine which block the element is in
auto entry = get_block_index_entry_for_index(index);
-
+
// Dequeue
auto block = entry->value.load(std::memory_order_relaxed);
auto& el = *((*block)[index]);
-
+
if (!MOODYCAMEL_NOEXCEPT_ASSIGN(T, T&&, element = std::move(el))) {
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
// Note: Acquiring the mutex with every dequeue instead of only when a block
// is released is very sub-optimal, but it is, after all, purely debug code.
debug::DebugLock lock(producer->mutex);
@@ -2491,7 +2579,7 @@ private:
index_t index;
BlockIndexEntry* entry;
ConcurrentQueue* parent;
-
+
~Guard()
{
(*block)[index]->~T();
@@ -2501,16 +2589,16 @@ private:
}
}
} guard = { block, index, entry, this->parent };
-
- element = std::move(el);
+
+ element = std::move(el); // NOLINT
}
else {
- element = std::move(el);
- el.~T();
-
+ element = std::move(el); // NOLINT
+ el.~T(); // NOLINT
+
if (block->ConcurrentQueue::Block::template set_empty<implicit_context>(index)) {
{
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
debug::DebugLock lock(mutex);
#endif
// Add the block back into the global free pool (and remove from block index)
@@ -2519,45 +2607,49 @@ private:
this->parent->add_block_to_free_list(block); // releases the above store
}
}
-
+
return true;
}
else {
this->dequeueOvercommit.fetch_add(1, std::memory_order_release);
}
}
-
+
return false;
}
-
+
+#ifdef _MSC_VER
+#pragma warning(push)
+#pragma warning(disable: 4706) // assignment within conditional expression
+#endif
template<AllocationMode allocMode, typename It>
bool enqueue_bulk(It itemFirst, size_t count)
{
// First, we need to make sure we have enough room to enqueue all of the elements;
// this means pre-allocating blocks and putting them in the block index (but only if
// all the allocations succeeded).
-
+
// Note that the tailBlock we start off with may not be owned by us any more;
// this happens if it was filled up exactly to the top (setting tailIndex to
// the first index of the next block which is not yet allocated), then dequeued
// completely (putting it on the free list) before we enqueue again.
-
+
index_t startTailIndex = this->tailIndex.load(std::memory_order_relaxed);
auto startBlock = this->tailBlock;
Block* firstAllocatedBlock = nullptr;
auto endBlock = this->tailBlock;
-
+
// Figure out how many blocks we'll need to allocate, and do so
size_t blockBaseDiff = ((startTailIndex + count - 1) & ~static_cast<index_t>(BLOCK_SIZE - 1)) - ((startTailIndex - 1) & ~static_cast<index_t>(BLOCK_SIZE - 1));
index_t currentTailIndex = (startTailIndex - 1) & ~static_cast<index_t>(BLOCK_SIZE - 1);
if (blockBaseDiff > 0) {
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
debug::DebugLock lock(mutex);
#endif
do {
blockBaseDiff -= static_cast<index_t>(BLOCK_SIZE);
currentTailIndex += static_cast<index_t>(BLOCK_SIZE);
-
+
// Find out where we'll be inserting this block in the block index
BlockIndexEntry* idxEntry = nullptr; // initialization here unnecessary but compiler can't always tell
Block* newBlock;
@@ -2565,6 +2657,7 @@ private:
auto head = this->headIndex.load(std::memory_order_relaxed);
assert(!details::circular_less_than<index_t>(currentTailIndex, head));
bool full = !details::circular_less_than<index_t>(head, currentTailIndex + BLOCK_SIZE) || (MAX_SUBQUEUE_SIZE != details::const_numeric_max<size_t>::value && (MAX_SUBQUEUE_SIZE == 0 || MAX_SUBQUEUE_SIZE - BLOCK_SIZE < currentTailIndex - head));
+
if (full || !(indexInserted = insert_block_index_entry<allocMode>(idxEntry, currentTailIndex)) || (newBlock = this->parent->ConcurrentQueue::template requisition_block<allocMode>()) == nullptr) {
// Index allocation or block allocation failed; revert any other allocations
// and index insertions done so far for this operation
@@ -2581,19 +2674,19 @@ private:
}
this->parent->add_blocks_to_free_list(firstAllocatedBlock);
this->tailBlock = startBlock;
-
+
return false;
}
-
-#if MCDBGQ_TRACKMEM
+
+#ifdef MCDBGQ_TRACKMEM
newBlock->owner = this;
#endif
newBlock->ConcurrentQueue::Block::template reset_empty<implicit_context>();
newBlock->next = nullptr;
-
+
// Insert the new block into the index
idxEntry->value.store(newBlock, std::memory_order_relaxed);
-
+
// Store the chain of blocks so that we can undo if later allocations fail,
// and so that we can find the blocks when we do the actual enqueueing
if ((startTailIndex & static_cast<index_t>(BLOCK_SIZE - 1)) != 0 || firstAllocatedBlock != nullptr) {
@@ -2605,7 +2698,7 @@ private:
firstAllocatedBlock = firstAllocatedBlock == nullptr ? newBlock : firstAllocatedBlock;
} while (blockBaseDiff > 0);
}
-
+
// Enqueue, one block at a time
index_t newTailIndex = startTailIndex + static_cast<index_t>(count);
currentTailIndex = startTailIndex;
@@ -2615,11 +2708,11 @@ private:
this->tailBlock = firstAllocatedBlock;
}
while (true) {
- auto stopIndex = (currentTailIndex & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
+ index_t stopIndex = (currentTailIndex & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
if (details::circular_less_than<index_t>(newTailIndex, stopIndex)) {
stopIndex = newTailIndex;
}
- if (MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (nullptr) T(details::deref_noexcept(itemFirst)))) {
+ MOODYCAMEL_CONSTEXPR_IF (MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (static_cast<T*>(nullptr)) T(details::deref_noexcept(itemFirst)))) {
while (currentTailIndex != stopIndex) {
new ((*this->tailBlock)[currentTailIndex++]) T(*itemFirst++);
}
@@ -2627,7 +2720,7 @@ private:
else {
MOODYCAMEL_TRY {
while (currentTailIndex != stopIndex) {
- new ((*this->tailBlock)[currentTailIndex]) T(details::nomove_if<(bool)!MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (nullptr) T(details::deref_noexcept(itemFirst)))>::eval(*itemFirst));
+ new ((*this->tailBlock)[currentTailIndex]) T(details::nomove_if<!MOODYCAMEL_NOEXCEPT_CTOR(T, decltype(*itemFirst), new (static_cast<T*>(nullptr)) T(details::deref_noexcept(itemFirst)))>::eval(*itemFirst));
++currentTailIndex;
++itemFirst;
}
@@ -2635,7 +2728,7 @@ private:
MOODYCAMEL_CATCH (...) {
auto constructedStopIndex = currentTailIndex;
auto lastBlockEnqueued = this->tailBlock;
-
+
if (!details::is_trivially_destructible<T>::value) {
auto block = startBlock;
if ((startTailIndex & static_cast<index_t>(BLOCK_SIZE - 1)) == 0) {
@@ -2656,7 +2749,7 @@ private:
block = block->next;
}
}
-
+
currentTailIndex = (startTailIndex - 1) & ~static_cast<index_t>(BLOCK_SIZE - 1);
for (auto block = firstAllocatedBlock; block != nullptr; block = block->next) {
currentTailIndex += static_cast<index_t>(BLOCK_SIZE);
@@ -2669,7 +2762,7 @@ private:
MOODYCAMEL_RETHROW;
}
}
-
+
if (this->tailBlock == endBlock) {
assert(currentTailIndex == newTailIndex);
break;
@@ -2679,7 +2772,10 @@ private:
this->tailIndex.store(newTailIndex, std::memory_order_release);
return true;
}
-
+#ifdef _MSC_VER
+#pragma warning(pop)
+#endif
+
template<typename It>
size_t dequeue_bulk(It& itemFirst, size_t max)
{
@@ -2689,10 +2785,9 @@ private:
if (details::circular_less_than<size_t>(0, desiredCount)) {
desiredCount = desiredCount < max ? desiredCount : max;
std::atomic_thread_fence(std::memory_order_acquire);
-
+
auto myDequeueCount = this->dequeueOptimisticCount.fetch_add(desiredCount, std::memory_order_relaxed);
- assert(overcommit <= myDequeueCount);
-
+
tail = this->tailIndex.load(std::memory_order_acquire);
auto actualCount = static_cast<size_t>(tail - (myDequeueCount - overcommit));
if (details::circular_less_than<size_t>(0, actualCount)) {
@@ -2700,20 +2795,20 @@ private:
if (actualCount < desiredCount) {
this->dequeueOvercommit.fetch_add(desiredCount - actualCount, std::memory_order_release);
}
-
+
// Get the first index. Note that since there's guaranteed to be at least actualCount elements, this
// will never exceed tail.
auto firstIndex = this->headIndex.fetch_add(actualCount, std::memory_order_acq_rel);
-
+
// Iterate the blocks and dequeue
auto index = firstIndex;
BlockIndexHeader* localBlockIndex;
auto indexIndex = get_block_index_index_for_index(index, localBlockIndex);
do {
auto blockStartIndex = index;
- auto endIndex = (index & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
+ index_t endIndex = (index & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
endIndex = details::circular_less_than<index_t>(firstIndex + static_cast<index_t>(actualCount), endIndex) ? firstIndex + static_cast<index_t>(actualCount) : endIndex;
-
+
auto entry = localBlockIndex->index[indexIndex];
auto block = entry->value.load(std::memory_order_relaxed);
if (MOODYCAMEL_NOEXCEPT_ASSIGN(T, T&&, details::deref_noexcept(itemFirst) = std::move((*(*block)[index])))) {
@@ -2741,27 +2836,27 @@ private:
while (index != endIndex) {
(*block)[index++]->~T();
}
-
+
if (block->ConcurrentQueue::Block::template set_many_empty<implicit_context>(blockStartIndex, static_cast<size_t>(endIndex - blockStartIndex))) {
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
debug::DebugLock lock(mutex);
#endif
entry->value.store(nullptr, std::memory_order_relaxed);
this->parent->add_block_to_free_list(block);
}
indexIndex = (indexIndex + 1) & (localBlockIndex->capacity - 1);
-
+
blockStartIndex = index;
endIndex = (index & ~static_cast<index_t>(BLOCK_SIZE - 1)) + static_cast<index_t>(BLOCK_SIZE);
endIndex = details::circular_less_than<index_t>(firstIndex + static_cast<index_t>(actualCount), endIndex) ? firstIndex + static_cast<index_t>(actualCount) : endIndex;
} while (index != firstIndex + actualCount);
-
+
MOODYCAMEL_RETHROW;
}
}
if (block->ConcurrentQueue::Block::template set_many_empty<implicit_context>(blockStartIndex, static_cast<size_t>(endIndex - blockStartIndex))) {
{
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
debug::DebugLock lock(mutex);
#endif
// Note that the set_many_empty above did a release, meaning that anybody who acquires the block
@@ -2772,27 +2867,27 @@ private:
}
indexIndex = (indexIndex + 1) & (localBlockIndex->capacity - 1);
} while (index != firstIndex + actualCount);
-
+
return actualCount;
}
else {
this->dequeueOvercommit.fetch_add(desiredCount, std::memory_order_release);
}
}
-
+
return 0;
}
-
+
private:
// The block size must be > 1, so any number with the low bit set is an invalid block base index
static const index_t INVALID_BLOCK_BASE = 1;
-
+
struct BlockIndexEntry
{
std::atomic<index_t> key;
std::atomic<Block*> value;
};
-
+
struct BlockIndexHeader
{
size_t capacity;
@@ -2801,7 +2896,7 @@ private:
BlockIndexEntry** index;
BlockIndexHeader* prev;
};
-
+
template<AllocationMode allocMode>
inline bool insert_block_index_entry(BlockIndexEntry*& idxEntry, index_t blockStartIndex)
{
@@ -2809,45 +2904,50 @@ private:
if (localBlockIndex == nullptr) {
return false; // this can happen if new_block_index failed in the constructor
}
- auto newTail = (localBlockIndex->tail.load(std::memory_order_relaxed) + 1) & (localBlockIndex->capacity - 1);
+ size_t newTail = (localBlockIndex->tail.load(std::memory_order_relaxed) + 1) & (localBlockIndex->capacity - 1);
idxEntry = localBlockIndex->index[newTail];
if (idxEntry->key.load(std::memory_order_relaxed) == INVALID_BLOCK_BASE ||
idxEntry->value.load(std::memory_order_relaxed) == nullptr) {
-
+
idxEntry->key.store(blockStartIndex, std::memory_order_relaxed);
localBlockIndex->tail.store(newTail, std::memory_order_release);
return true;
}
-
+
// No room in the old block index, try to allocate another one!
- if (allocMode == CannotAlloc || !new_block_index()) {
+ MOODYCAMEL_CONSTEXPR_IF (allocMode == CannotAlloc) {
return false;
}
- localBlockIndex = blockIndex.load(std::memory_order_relaxed);
- newTail = (localBlockIndex->tail.load(std::memory_order_relaxed) + 1) & (localBlockIndex->capacity - 1);
- idxEntry = localBlockIndex->index[newTail];
- assert(idxEntry->key.load(std::memory_order_relaxed) == INVALID_BLOCK_BASE);
- idxEntry->key.store(blockStartIndex, std::memory_order_relaxed);
- localBlockIndex->tail.store(newTail, std::memory_order_release);
- return true;
+ else if (!new_block_index()) {
+ return false;
+ }
+ else {
+ localBlockIndex = blockIndex.load(std::memory_order_relaxed);
+ newTail = (localBlockIndex->tail.load(std::memory_order_relaxed) + 1) & (localBlockIndex->capacity - 1);
+ idxEntry = localBlockIndex->index[newTail];
+ assert(idxEntry->key.load(std::memory_order_relaxed) == INVALID_BLOCK_BASE);
+ idxEntry->key.store(blockStartIndex, std::memory_order_relaxed);
+ localBlockIndex->tail.store(newTail, std::memory_order_release);
+ return true;
+ }
}
-
+
inline void rewind_block_index_tail()
{
auto localBlockIndex = blockIndex.load(std::memory_order_relaxed);
localBlockIndex->tail.store((localBlockIndex->tail.load(std::memory_order_relaxed) - 1) & (localBlockIndex->capacity - 1), std::memory_order_relaxed);
}
-
+
inline BlockIndexEntry* get_block_index_entry_for_index(index_t index) const
{
BlockIndexHeader* localBlockIndex;
auto idx = get_block_index_index_for_index(index, localBlockIndex);
return localBlockIndex->index[idx];
}
-
+
inline size_t get_block_index_index_for_index(index_t index, BlockIndexHeader*& localBlockIndex) const
{
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
debug::DebugLock lock(mutex);
#endif
index &= ~static_cast<index_t>(BLOCK_SIZE - 1);
@@ -2857,12 +2957,12 @@ private:
assert(tailBase != INVALID_BLOCK_BASE);
// Note: Must use division instead of shift because the index may wrap around, causing a negative
// offset, whose negativity we want to preserve
- auto offset = static_cast<size_t>(static_cast<typename std::make_signed<index_t>::type>(index - tailBase) / BLOCK_SIZE);
+ auto offset = static_cast<size_t>(static_cast<typename std::make_signed<index_t>::type>(index - tailBase) / static_cast<typename std::make_signed<index_t>::type>(BLOCK_SIZE));
size_t idx = (tail + offset) & (localBlockIndex->capacity - 1);
assert(localBlockIndex->index[idx]->key.load(std::memory_order_relaxed) == index && localBlockIndex->index[idx]->value.load(std::memory_order_relaxed) != nullptr);
return idx;
}
-
+
bool new_block_index()
{
auto prev = blockIndex.load(std::memory_order_relaxed);
@@ -2875,7 +2975,7 @@ private:
if (raw == nullptr) {
return false;
}
-
+
auto header = new (raw) BlockIndexHeader;
auto entries = reinterpret_cast<BlockIndexEntry*>(details::align_for<BlockIndexEntry>(raw + sizeof(BlockIndexHeader)));
auto index = reinterpret_cast<BlockIndexEntry**>(details::align_for<BlockIndexEntry*>(reinterpret_cast<char*>(entries) + sizeof(BlockIndexEntry) * entryCount));
@@ -2899,14 +2999,14 @@ private:
header->index = index;
header->capacity = nextBlockIndexCapacity;
header->tail.store((prevCapacity - 1) & (nextBlockIndexCapacity - 1), std::memory_order_relaxed);
-
+
blockIndex.store(header, std::memory_order_release);
-
+
nextBlockIndexCapacity <<= 1;
-
+
return true;
}
-
+
private:
size_t nextBlockIndexCapacity;
std::atomic<BlockIndexHeader*> blockIndex;
@@ -2916,26 +3016,26 @@ private:
details::ThreadExitListener threadExitListener;
private:
#endif
-
+
#ifdef MOODYCAMEL_QUEUE_INTERNAL_DEBUG
public:
ImplicitProducer* nextImplicitProducer;
private:
#endif
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODBLOCKINDEX
mutable debug::DebugMutex mutex;
#endif
-#if MCDBGQ_TRACKMEM
+#ifdef MCDBGQ_TRACKMEM
friend struct MemStats;
#endif
};
-
-
+
+
//////////////////////////////////
// Block pool manipulation
//////////////////////////////////
-
+
void populate_initial_block_list(size_t blockCount)
{
initialBlockPoolSize = blockCount;
@@ -2943,7 +3043,7 @@ private:
initialBlockPool = nullptr;
return;
}
-
+
initialBlockPool = create_array<Block>(blockCount);
if (initialBlockPool == nullptr) {
initialBlockPoolSize = 0;
@@ -2952,26 +3052,31 @@ private:
initialBlockPool[i].dynamicallyAllocated = false;
}
}
-
+
inline Block* try_get_block_from_initial_pool()
{
if (initialBlockPoolIndex.load(std::memory_order_relaxed) >= initialBlockPoolSize) {
return nullptr;
}
-
+
auto index = initialBlockPoolIndex.fetch_add(1, std::memory_order_relaxed);
-
+
return index < initialBlockPoolSize ? (initialBlockPool + index) : nullptr;
}
-
+
inline void add_block_to_free_list(Block* block)
{
-#if MCDBGQ_TRACKMEM
+#ifdef MCDBGQ_TRACKMEM
block->owner = nullptr;
#endif
- freeList.add(block);
+ if (!Traits::RECYCLE_ALLOCATED_BLOCKS && block->dynamicallyAllocated) {
+ destroy(block);
+ }
+ else {
+ freeList.add(block);
+ }
}
-
+
inline void add_blocks_to_free_list(Block* block)
{
while (block != nullptr) {
@@ -2980,12 +3085,12 @@ private:
block = next;
}
}
-
+
inline Block* try_get_block_from_free_list()
{
return freeList.try_get();
}
-
+
// Gets a free block from one of the memory pools, or allocates a new one (if applicable)
template<AllocationMode canAlloc>
Block* requisition_block()
@@ -2994,21 +3099,22 @@ private:
if (block != nullptr) {
return block;
}
-
+
block = try_get_block_from_free_list();
if (block != nullptr) {
return block;
}
-
- if (canAlloc == CanAlloc) {
+
+ MOODYCAMEL_CONSTEXPR_IF (canAlloc == CanAlloc) {
return create<Block>();
}
-
- return nullptr;
+ else {
+ return nullptr;
+ }
}
-
-#if MCDBGQ_TRACKMEM
+
+#ifdef MCDBGQ_TRACKMEM
public:
struct MemStats {
size_t allocatedBlocks;
@@ -3023,28 +3129,28 @@ private:
size_t queueClassBytes;
size_t implicitBlockIndexBytes;
size_t explicitBlockIndexBytes;
-
+
friend class ConcurrentQueue;
-
+
private:
static MemStats getFor(ConcurrentQueue* q)
{
MemStats stats = { 0 };
-
+
stats.elementsEnqueued = q->size_approx();
-
+
auto block = q->freeList.head_unsafe();
while (block != nullptr) {
++stats.allocatedBlocks;
++stats.freeBlocks;
block = block->freeListNext.load(std::memory_order_relaxed);
}
-
+
for (auto ptr = q->producerListTail.load(std::memory_order_acquire); ptr != nullptr; ptr = ptr->next_prod()) {
bool implicit = dynamic_cast<ImplicitProducer*>(ptr) != nullptr;
stats.implicitProducers += implicit ? 1 : 0;
stats.explicitProducers += implicit ? 0 : 1;
-
+
if (implicit) {
auto prod = static_cast<ImplicitProducer*>(ptr);
stats.queueClassBytes += sizeof(ImplicitProducer);
@@ -3092,18 +3198,18 @@ private:
}
}
}
-
+
auto freeOnInitialPool = q->initialBlockPoolIndex.load(std::memory_order_relaxed) >= q->initialBlockPoolSize ? 0 : q->initialBlockPoolSize - q->initialBlockPoolIndex.load(std::memory_order_relaxed);
stats.allocatedBlocks += freeOnInitialPool;
stats.freeBlocks += freeOnInitialPool;
-
+
stats.blockClassBytes = sizeof(Block) * stats.allocatedBlocks;
stats.queueClassBytes += sizeof(ConcurrentQueue);
-
+
return stats;
}
};
-
+
// For debugging only. Not thread-safe.
MemStats getMemStats()
{
@@ -3112,21 +3218,15 @@ private:
private:
friend struct MemStats;
#endif
-
-
+
+
//////////////////////////////////
// Producer list manipulation
- //////////////////////////////////
-
+ //////////////////////////////////
+
ProducerBase* recycle_or_create_producer(bool isExplicit)
{
- bool recycled;
- return recycle_or_create_producer(isExplicit, recycled);
- }
-
- ProducerBase* recycle_or_create_producer(bool isExplicit, bool& recycled)
- {
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODHASH
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODHASH
debug::DebugLock lock(implicitProdMutex);
#endif
// Try to re-use one first
@@ -3135,31 +3235,29 @@ private:
bool expected = true;
if (ptr->inactive.compare_exchange_strong(expected, /* desired */ false, std::memory_order_acquire, std::memory_order_relaxed)) {
// We caught one! It's been marked as activated, the caller can have it
- recycled = true;
return ptr;
}
}
}
-
- recycled = false;
+
return add_producer(isExplicit ? static_cast<ProducerBase*>(create<ExplicitProducer>(this)) : create<ImplicitProducer>(this));
}
-
+
ProducerBase* add_producer(ProducerBase* producer)
{
// Handle failed memory allocation
if (producer == nullptr) {
return nullptr;
}
-
+
producerCount.fetch_add(1, std::memory_order_relaxed);
-
+
// Add it to the lock-free list
auto prevTail = producerListTail.load(std::memory_order_relaxed);
do {
producer->next = prevTail;
} while (!producerListTail.compare_exchange_weak(prevTail, producer, std::memory_order_release, std::memory_order_relaxed));
-
+
#ifdef MOODYCAMEL_QUEUE_INTERNAL_DEBUG
if (producer->isExplicit) {
auto prevTailExplicit = explicitProducers.load(std::memory_order_relaxed);
@@ -3174,10 +3272,10 @@ private:
} while (!implicitProducers.compare_exchange_weak(prevTailImplicit, static_cast<ImplicitProducer*>(producer), std::memory_order_release, std::memory_order_relaxed));
}
#endif
-
+
return producer;
}
-
+
void reown_producers()
{
// After another instance is moved-into/swapped-with this one, all the
@@ -3187,31 +3285,31 @@ private:
ptr->parent = this;
}
}
-
-
+
+
//////////////////////////////////
// Implicit producer hash
//////////////////////////////////
-
+
struct ImplicitProducerKVP
{
std::atomic<details::thread_id_t> key;
ImplicitProducer* value; // No need for atomicity since it's only read by the thread that sets it in the first place
-
+
ImplicitProducerKVP() : value(nullptr) { }
-
+
ImplicitProducerKVP(ImplicitProducerKVP&& other) MOODYCAMEL_NOEXCEPT
{
key.store(other.key.load(std::memory_order_relaxed), std::memory_order_relaxed);
value = other.value;
}
-
+
inline ImplicitProducerKVP& operator=(ImplicitProducerKVP&& other) MOODYCAMEL_NOEXCEPT
{
swap(other);
return *this;
}
-
+
inline void swap(ImplicitProducerKVP& other) MOODYCAMEL_NOEXCEPT
{
if (this != &other) {
@@ -3220,93 +3318,100 @@ private:
}
}
};
-
+
template<typename XT, typename XTraits>
friend void moodycamel::swap(typename ConcurrentQueue<XT, XTraits>::ImplicitProducerKVP&, typename ConcurrentQueue<XT, XTraits>::ImplicitProducerKVP&) MOODYCAMEL_NOEXCEPT;
-
+
struct ImplicitProducerHash
{
size_t capacity;
ImplicitProducerKVP* entries;
ImplicitProducerHash* prev;
};
-
+
inline void populate_initial_implicit_producer_hash()
{
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return;
-
- implicitProducerHashCount.store(0, std::memory_order_relaxed);
- auto hash = &initialImplicitProducerHash;
- hash->capacity = INITIAL_IMPLICIT_PRODUCER_HASH_SIZE;
- hash->entries = &initialImplicitProducerHashEntries[0];
- for (size_t i = 0; i != INITIAL_IMPLICIT_PRODUCER_HASH_SIZE; ++i) {
- initialImplicitProducerHashEntries[i].key.store(details::invalid_thread_id, std::memory_order_relaxed);
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) {
+ return;
+ }
+ else {
+ implicitProducerHashCount.store(0, std::memory_order_relaxed);
+ auto hash = &initialImplicitProducerHash;
+ hash->capacity = INITIAL_IMPLICIT_PRODUCER_HASH_SIZE;
+ hash->entries = &initialImplicitProducerHashEntries[0];
+ for (size_t i = 0; i != INITIAL_IMPLICIT_PRODUCER_HASH_SIZE; ++i) {
+ initialImplicitProducerHashEntries[i].key.store(details::invalid_thread_id, std::memory_order_relaxed);
+ }
+ hash->prev = nullptr;
+ implicitProducerHash.store(hash, std::memory_order_relaxed);
}
- hash->prev = nullptr;
- implicitProducerHash.store(hash, std::memory_order_relaxed);
}
-
+
void swap_implicit_producer_hashes(ConcurrentQueue& other)
{
- if (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) return;
-
- // Swap (assumes our implicit producer hash is initialized)
- initialImplicitProducerHashEntries.swap(other.initialImplicitProducerHashEntries);
- initialImplicitProducerHash.entries = &initialImplicitProducerHashEntries[0];
- other.initialImplicitProducerHash.entries = &other.initialImplicitProducerHashEntries[0];
-
- details::swap_relaxed(implicitProducerHashCount, other.implicitProducerHashCount);
-
- details::swap_relaxed(implicitProducerHash, other.implicitProducerHash);
- if (implicitProducerHash.load(std::memory_order_relaxed) == &other.initialImplicitProducerHash) {
- implicitProducerHash.store(&initialImplicitProducerHash, std::memory_order_relaxed);
+ MOODYCAMEL_CONSTEXPR_IF (INITIAL_IMPLICIT_PRODUCER_HASH_SIZE == 0) {
+ return;
}
else {
- ImplicitProducerHash* hash;
- for (hash = implicitProducerHash.load(std::memory_order_relaxed); hash->prev != &other.initialImplicitProducerHash; hash = hash->prev) {
- continue;
+ // Swap (assumes our implicit producer hash is initialized)
+ initialImplicitProducerHashEntries.swap(other.initialImplicitProducerHashEntries);
+ initialImplicitProducerHash.entries = &initialImplicitProducerHashEntries[0];
+ other.initialImplicitProducerHash.entries = &other.initialImplicitProducerHashEntries[0];
+
+ details::swap_relaxed(implicitProducerHashCount, other.implicitProducerHashCount);
+
+ details::swap_relaxed(implicitProducerHash, other.implicitProducerHash);
+ if (implicitProducerHash.load(std::memory_order_relaxed) == &other.initialImplicitProducerHash) {
+ implicitProducerHash.store(&initialImplicitProducerHash, std::memory_order_relaxed);
}
- hash->prev = &initialImplicitProducerHash;
- }
- if (other.implicitProducerHash.load(std::memory_order_relaxed) == &initialImplicitProducerHash) {
- other.implicitProducerHash.store(&other.initialImplicitProducerHash, std::memory_order_relaxed);
- }
- else {
- ImplicitProducerHash* hash;
- for (hash = other.implicitProducerHash.load(std::memory_order_relaxed); hash->prev != &initialImplicitProducerHash; hash = hash->prev) {
- continue;
+ else {
+ ImplicitProducerHash* hash;
+ for (hash = implicitProducerHash.load(std::memory_order_relaxed); hash->prev != &other.initialImplicitProducerHash; hash = hash->prev) {
+ continue;
+ }
+ hash->prev = &initialImplicitProducerHash;
+ }
+ if (other.implicitProducerHash.load(std::memory_order_relaxed) == &initialImplicitProducerHash) {
+ other.implicitProducerHash.store(&other.initialImplicitProducerHash, std::memory_order_relaxed);
+ }
+ else {
+ ImplicitProducerHash* hash;
+ for (hash = other.implicitProducerHash.load(std::memory_order_relaxed); hash->prev != &initialImplicitProducerHash; hash = hash->prev) {
+ continue;
+ }
+ hash->prev = &other.initialImplicitProducerHash;
}
- hash->prev = &other.initialImplicitProducerHash;
}
}
-
+
// Only fails (returns nullptr) if memory allocation fails
ImplicitProducer* get_or_add_implicit_producer()
{
// Note that since the data is essentially thread-local (key is thread ID),
// there's a reduced need for fences (memory ordering is already consistent
// for any individual thread), except for the current table itself.
-
+
// Start by looking for the thread ID in the current and all previous hash tables.
// If it's not found, it must not be in there yet, since this same thread would
// have added it previously to one of the tables that we traversed.
-
+
// Code and algorithm adapted from http://preshing.com/20130605/the-worlds-simplest-lock-free-hash-table
-
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODHASH
+
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODHASH
debug::DebugLock lock(implicitProdMutex);
#endif
-
+
auto id = details::thread_id();
auto hashedId = details::hash_thread_id(id);
-
+
auto mainHash = implicitProducerHash.load(std::memory_order_acquire);
+ assert(mainHash != nullptr); // silence clang-tidy and MSVC warnings (hash cannot be null)
for (auto hash = mainHash; hash != nullptr; hash = hash->prev) {
// Look for the id in this hash
auto index = hashedId;
while (true) { // Not an infinite loop because at least one slot is free in the hash table
- index &= hash->capacity - 1;
-
+ index &= hash->capacity - 1u;
+
auto probedKey = hash->entries[index].key.load(std::memory_order_relaxed);
if (probedKey == id) {
// Found it! If we had to search several hashes deep, though, we should lazily add it
@@ -3318,15 +3423,14 @@ private:
if (hash != mainHash) {
index = hashedId;
while (true) {
- index &= mainHash->capacity - 1;
- probedKey = mainHash->entries[index].key.load(std::memory_order_relaxed);
+ index &= mainHash->capacity - 1u;
auto empty = details::invalid_thread_id;
#ifdef MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED
auto reusable = details::invalid_thread_id2;
- if ((probedKey == empty && mainHash->entries[index].key.compare_exchange_strong(empty, id, std::memory_order_relaxed, std::memory_order_relaxed)) ||
- (probedKey == reusable && mainHash->entries[index].key.compare_exchange_strong(reusable, id, std::memory_order_acquire, std::memory_order_acquire))) {
+ if (mainHash->entries[index].key.compare_exchange_strong(empty, id, std::memory_order_seq_cst, std::memory_order_relaxed) ||
+ mainHash->entries[index].key.compare_exchange_strong(reusable, id, std::memory_order_seq_cst, std::memory_order_relaxed)) {
#else
- if ((probedKey == empty && mainHash->entries[index].key.compare_exchange_strong(empty, id, std::memory_order_relaxed, std::memory_order_relaxed))) {
+ if (mainHash->entries[index].key.compare_exchange_strong(empty, id, std::memory_order_seq_cst, std::memory_order_relaxed)) {
#endif
mainHash->entries[index].value = value;
break;
@@ -3334,7 +3438,7 @@ private:
++index;
}
}
-
+
return value;
}
if (probedKey == details::invalid_thread_id) {
@@ -3343,10 +3447,11 @@ private:
++index;
}
}
-
+
// Insert!
auto newCount = 1 + implicitProducerHashCount.fetch_add(1, std::memory_order_relaxed);
while (true) {
+ // NOLINTNEXTLINE(clang-analyzer-core.NullDereference)
if (newCount >= (mainHash->capacity >> 1) && !implicitProducerHashResizeInProgress.test_and_set(std::memory_order_acquire)) {
// We've acquired the resize lock, try to allocate a bigger hash table.
// Note the acquire fence synchronizes with the release fence at the end of this block, and hence when
@@ -3354,20 +3459,20 @@ private:
// locked block).
mainHash = implicitProducerHash.load(std::memory_order_acquire);
if (newCount >= (mainHash->capacity >> 1)) {
- auto newCapacity = mainHash->capacity << 1;
+ size_t newCapacity = mainHash->capacity << 1;
while (newCount >= (newCapacity >> 1)) {
newCapacity <<= 1;
}
auto raw = static_cast<char*>((Traits::malloc)(sizeof(ImplicitProducerHash) + std::alignment_of<ImplicitProducerKVP>::value - 1 + sizeof(ImplicitProducerKVP) * newCapacity));
if (raw == nullptr) {
// Allocation failed
- implicitProducerHashCount.fetch_add(-1, std::memory_order_relaxed);
+ implicitProducerHashCount.fetch_sub(1, std::memory_order_relaxed);
implicitProducerHashResizeInProgress.clear(std::memory_order_relaxed);
return nullptr;
}
-
+
auto newHash = new (raw) ImplicitProducerHash;
- newHash->capacity = newCapacity;
+ newHash->capacity = static_cast<size_t>(newCapacity);
newHash->entries = reinterpret_cast<ImplicitProducerKVP*>(details::align_for<ImplicitProducerKVP>(raw + sizeof(ImplicitProducerHash)));
for (size_t i = 0; i != newCapacity; ++i) {
new (newHash->entries + i) ImplicitProducerKVP;
@@ -3382,40 +3487,36 @@ private:
implicitProducerHashResizeInProgress.clear(std::memory_order_release);
}
}
-
+
// If it's < three-quarters full, add to the old one anyway so that we don't have to wait for the next table
// to finish being allocated by another thread (and if we just finished allocating above, the condition will
// always be true)
if (newCount < (mainHash->capacity >> 1) + (mainHash->capacity >> 2)) {
- bool recycled;
- auto producer = static_cast<ImplicitProducer*>(recycle_or_create_producer(false, recycled));
+ auto producer = static_cast<ImplicitProducer*>(recycle_or_create_producer(false));
if (producer == nullptr) {
- implicitProducerHashCount.fetch_add(-1, std::memory_order_relaxed);
+ implicitProducerHashCount.fetch_sub(1, std::memory_order_relaxed);
return nullptr;
}
- if (recycled) {
- implicitProducerHashCount.fetch_add(-1, std::memory_order_relaxed);
- }
-
+
#ifdef MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED
producer->threadExitListener.callback = &ConcurrentQueue::implicit_producer_thread_exited_callback;
producer->threadExitListener.userData = producer;
details::ThreadExitNotifier::subscribe(&producer->threadExitListener);
#endif
-
+
auto index = hashedId;
while (true) {
- index &= mainHash->capacity - 1;
- auto probedKey = mainHash->entries[index].key.load(std::memory_order_relaxed);
-
+ index &= mainHash->capacity - 1u;
auto empty = details::invalid_thread_id;
#ifdef MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED
auto reusable = details::invalid_thread_id2;
- if ((probedKey == empty && mainHash->entries[index].key.compare_exchange_strong(empty, id, std::memory_order_relaxed, std::memory_order_relaxed)) ||
- (probedKey == reusable && mainHash->entries[index].key.compare_exchange_strong(reusable, id, std::memory_order_acquire, std::memory_order_acquire))) {
-#else
- if ((probedKey == empty && mainHash->entries[index].key.compare_exchange_strong(empty, id, std::memory_order_relaxed, std::memory_order_relaxed))) {
+ if (mainHash->entries[index].key.compare_exchange_strong(reusable, id, std::memory_order_seq_cst, std::memory_order_relaxed)) {
+ implicitProducerHashCount.fetch_sub(1, std::memory_order_relaxed); // already counted as a used slot
+ mainHash->entries[index].value = producer;
+ break;
+ }
#endif
+ if (mainHash->entries[index].key.compare_exchange_strong(empty, id, std::memory_order_seq_cst, std::memory_order_relaxed)) {
mainHash->entries[index].value = producer;
break;
}
@@ -3423,22 +3524,19 @@ private:
}
return producer;
}
-
+
// Hmm, the old hash is quite full and somebody else is busy allocating a new one.
// We need to wait for the allocating thread to finish (if it succeeds, we add, if not,
// we try to allocate ourselves).
mainHash = implicitProducerHash.load(std::memory_order_acquire);
}
}
-
+
#ifdef MOODYCAMEL_CPP11_THREAD_LOCAL_SUPPORTED
void implicit_producer_thread_exited(ImplicitProducer* producer)
{
- // Remove from thread exit listeners
- details::ThreadExitNotifier::unsubscribe(&producer->threadExitListener);
-
// Remove from hash
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODHASH
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODHASH
debug::DebugLock lock(implicitProdMutex);
#endif
auto hash = implicitProducerHash.load(std::memory_order_acquire);
@@ -3446,26 +3544,25 @@ private:
auto id = details::thread_id();
auto hashedId = details::hash_thread_id(id);
details::thread_id_t probedKey;
-
+
// We need to traverse all the hashes just in case other threads aren't on the current one yet and are
// trying to add an entry thinking there's a free slot (because they reused a producer)
for (; hash != nullptr; hash = hash->prev) {
auto index = hashedId;
do {
- index &= hash->capacity - 1;
- probedKey = hash->entries[index].key.load(std::memory_order_relaxed);
- if (probedKey == id) {
- hash->entries[index].key.store(details::invalid_thread_id2, std::memory_order_release);
+ index &= hash->capacity - 1u;
+ probedKey = id;
+ if (hash->entries[index].key.compare_exchange_strong(probedKey, details::invalid_thread_id2, std::memory_order_seq_cst, std::memory_order_relaxed)) {
break;
}
++index;
} while (probedKey != details::invalid_thread_id); // Can happen if the hash has changed but we weren't put back in it yet, or if we weren't added to this hash in the first place
}
-
+
// Mark the queue as being recyclable
producer->inactive.store(true, std::memory_order_release);
}
-
+
static void implicit_producer_thread_exited_callback(void* userData)
{
auto producer = static_cast<ImplicitProducer*>(userData);
@@ -3473,88 +3570,109 @@ private:
queue->implicit_producer_thread_exited(producer);
}
#endif
-
+
//////////////////////////////////
// Utility functions
//////////////////////////////////
-
+
+ template<typename TAlign>
+ static inline void* aligned_malloc(size_t size)
+ {
+ MOODYCAMEL_CONSTEXPR_IF (std::alignment_of<TAlign>::value <= std::alignment_of<details::max_align_t>::value)
+ return (Traits::malloc)(size);
+ else {
+ size_t alignment = std::alignment_of<TAlign>::value;
+ void* raw = (Traits::malloc)(size + alignment - 1 + sizeof(void*));
+ if (!raw)
+ return nullptr;
+ char* ptr = details::align_for<TAlign>(reinterpret_cast<char*>(raw) + sizeof(void*));
+ *(reinterpret_cast<void**>(ptr) - 1) = raw;
+ return ptr;
+ }
+ }
+
+ template<typename TAlign>
+ static inline void aligned_free(void* ptr)
+ {
+ MOODYCAMEL_CONSTEXPR_IF (std::alignment_of<TAlign>::value <= std::alignment_of<details::max_align_t>::value)
+ return (Traits::free)(ptr);
+ else
+ (Traits::free)(ptr ? *(reinterpret_cast<void**>(ptr) - 1) : nullptr);
+ }
+
template<typename U>
static inline U* create_array(size_t count)
{
assert(count > 0);
- auto p = static_cast<U*>((Traits::malloc)(sizeof(U) * count));
- if (p == nullptr) {
+ U* p = static_cast<U*>(aligned_malloc<U>(sizeof(U) * count));
+ if (p == nullptr)
return nullptr;
- }
-
- for (size_t i = 0; i != count; ++i) {
+
+ for (size_t i = 0; i != count; ++i)
new (p + i) U();
- }
return p;
}
-
+
template<typename U>
static inline void destroy_array(U* p, size_t count)
{
if (p != nullptr) {
assert(count > 0);
- for (size_t i = count; i != 0; ) {
+ for (size_t i = count; i != 0; )
(p + --i)->~U();
- }
- (Traits::free)(p);
}
+ aligned_free<U>(p);
}
-
+
template<typename U>
static inline U* create()
{
- auto p = (Traits::malloc)(sizeof(U));
+ void* p = aligned_malloc<U>(sizeof(U));
return p != nullptr ? new (p) U : nullptr;
}
-
+
template<typename U, typename A1>
static inline U* create(A1&& a1)
{
- auto p = (Traits::malloc)(sizeof(U));
+ void* p = aligned_malloc<U>(sizeof(U));
return p != nullptr ? new (p) U(std::forward<A1>(a1)) : nullptr;
}
-
+
template<typename U>
static inline void destroy(U* p)
{
- if (p != nullptr) {
+ if (p != nullptr)
p->~U();
- }
- (Traits::free)(p);
+ aligned_free<U>(p);
}
private:
std::atomic<ProducerBase*> producerListTail;
std::atomic<std::uint32_t> producerCount;
-
+
std::atomic<size_t> initialBlockPoolIndex;
Block* initialBlockPool;
size_t initialBlockPoolSize;
-
-#if !MCDBGQ_USEDEBUGFREELIST
+
+#ifndef MCDBGQ_USEDEBUGFREELIST
FreeList<Block> freeList;
#else
debug::DebugFreeList<Block> freeList;
#endif
-
+
std::atomic<ImplicitProducerHash*> implicitProducerHash;
std::atomic<size_t> implicitProducerHashCount; // Number of slots logically used
ImplicitProducerHash initialImplicitProducerHash;
std::array<ImplicitProducerKVP, INITIAL_IMPLICIT_PRODUCER_HASH_SIZE> initialImplicitProducerHashEntries;
std::atomic_flag implicitProducerHashResizeInProgress;
-
+
std::atomic<std::uint32_t> nextExplicitConsumerId;
std::atomic<std::uint32_t> globalExplicitConsumerOffset;
-
-#if MCDBGQ_NOLOCKFREE_IMPLICITPRODHASH
+
+#ifdef MCDBGQ_NOLOCKFREE_IMPLICITPRODHASH
debug::DebugMutex implicitProdMutex;
#endif
-
+
#ifdef MOODYCAMEL_QUEUE_INTERNAL_DEBUG
std::atomic<ExplicitProducer*> explicitProducers;
std::atomic<ImplicitProducer*> implicitProducers;
@@ -3585,7 +3703,7 @@ ConsumerToken::ConsumerToken(ConcurrentQueue<T, Traits>& queue)
: itemsConsumedFromCurrent(0), currentProducer(nullptr), desiredProducer(nullptr)
{
initialOffset = queue.nextExplicitConsumerId.fetch_add(1, std::memory_order_release);
- lastKnownGlobalOffset = -1;
+ lastKnownGlobalOffset = static_cast<std::uint32_t>(-1);
}
template<typename T, typename Traits>
@@ -3593,7 +3711,7 @@ ConsumerToken::ConsumerToken(BlockingConcurrentQueue<T, Traits>& queue)
: itemsConsumedFromCurrent(0), currentProducer(nullptr), desiredProducer(nullptr)
{
initialOffset = reinterpret_cast<ConcurrentQueue<T, Traits>*>(&queue)->nextExplicitConsumerId.fetch_add(1, std::memory_order_release);
- lastKnownGlobalOffset = -1;
+ lastKnownGlobalOffset = static_cast<std::uint32_t>(-1);
}
template<typename T, typename Traits>
@@ -3620,6 +3738,10 @@ inline void swap(typename ConcurrentQueue<T, Traits>::ImplicitProducerKVP& a, ty
}
-#if defined(__GNUC__)
+#if defined(_MSC_VER) && (!defined(_HAS_CXX17) || !_HAS_CXX17)
+#pragma warning(pop)
+#endif
+
+#if defined(__GNUC__) && !defined(__INTEL_COMPILER)
#pragma GCC diagnostic pop
#endif
(nifi-minifi-cpp) 16/37: Linter fixes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 59b524ba28df749b57c2670911ffe6b7cf09250e
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon May 12 17:08:04 2025 +0200
Linter fixes
---
extensions/python/PythonCreator.h | 2 +-
extensions/sftp/processors/SFTPProcessorBase.h | 2 +-
.../tests/unit/DefragmentTextTests.cpp | 2 +-
libminifi/src/core/ClassLoader.cpp | 31 +++++++++++-----------
minifi-api/include/minifi-cpp/core/ClassLoader.h | 14 +++++-----
5 files changed, 26 insertions(+), 25 deletions(-)
diff --git a/extensions/python/PythonCreator.h b/extensions/python/PythonCreator.h
index 3cc2c1284..362f737db 100644
--- a/extensions/python/PythonCreator.h
+++ b/extensions/python/PythonCreator.h
@@ -60,7 +60,7 @@ class DummyLogger : public core::logging::Logger {
bool should_log(core::logging::LOG_LEVEL /*level*/) override {
return false;
}
- [[nodiscard]] virtual core::logging::LOG_LEVEL level() const override {
+ [[nodiscard]] core::logging::LOG_LEVEL level() const override {
return core::logging::LOG_LEVEL::off;
}
diff --git a/extensions/sftp/processors/SFTPProcessorBase.h b/extensions/sftp/processors/SFTPProcessorBase.h
index 628fd652e..a63c57d71 100644
--- a/extensions/sftp/processors/SFTPProcessorBase.h
+++ b/extensions/sftp/processors/SFTPProcessorBase.h
@@ -40,7 +40,7 @@ namespace org::apache::nifi::minifi::processors {
class SFTPProcessorBase : public core::ProcessorImpl {
public:
- SFTPProcessorBase(core::ProcessorMetadata info);
+ explicit SFTPProcessorBase(core::ProcessorMetadata info);
~SFTPProcessorBase() override;
static constexpr std::string_view PROXY_TYPE_DIRECT = "DIRECT";
diff --git a/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp b/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
index 49a22a8cc..1fb68b0c4 100644
--- a/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
+++ b/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
@@ -39,7 +39,7 @@ TEST_CASE("DefragmentText Single source tests", "[defragmenttextsinglesource]")
auto plan = testController.createPlan();
auto write_to_flow_file = plan->addProcessor<WriteToFlowFileTestProcessor>("write_to_flow_file");
auto defrag_text_flow_files = plan->addProcessor<DefragmentText>("defrag_text_flow_files");
- auto read_from_success_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>( "read_from_success_relationship");
+ auto read_from_success_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_success_relationship");
auto read_from_failure_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_failure_relationship");
plan->addConnection(write_to_flow_file, WriteToFlowFileTestProcessor::Success, defrag_text_flow_files);
diff --git a/libminifi/src/core/ClassLoader.cpp b/libminifi/src/core/ClassLoader.cpp
index a316887e6..b7076b369 100644
--- a/libminifi/src/core/ClassLoader.cpp
+++ b/libminifi/src/core/ClassLoader.cpp
@@ -31,7 +31,12 @@ class ClassLoaderImpl : public ClassLoader {
public:
explicit ClassLoaderImpl(std::string name = "/");
- ClassLoader& getClassLoader(const std::string& child_name) override;
+ ClassLoaderImpl(const ClassLoaderImpl&) = delete;
+ ClassLoaderImpl(ClassLoaderImpl&&) = delete;
+ ClassLoaderImpl& operator=(const ClassLoaderImpl&) = delete;
+ ClassLoaderImpl& operator=(ClassLoaderImpl&&) = delete;
+
+ [[nodiscard]] ClassLoader& getClassLoader(const std::string& child_name) override;
void registerClass(const std::string &clazz, std::unique_ptr<ObjectFactory> factory) override;
@@ -39,25 +44,21 @@ class ClassLoaderImpl : public ClassLoader {
void unregisterClass(const std::string& clazz) override;
- std::optional<std::string> getGroupForClass(const std::string &class_name) const override;
+ [[nodiscard]] std::optional<std::string> getGroupForClass(const std::string &class_name) const override;
- std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) override;
+ [[nodiscard]] std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) override;
- std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) override;
+ [[nodiscard]] std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) override;
- CoreComponent* instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) override;
+ [[nodiscard]] CoreComponent* instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) override;
~ClassLoaderImpl() override = default;
private:
std::map<std::string, std::unique_ptr<ObjectFactory>> loaded_factories_;
-
std::map<std::string, ClassLoaderImpl> class_loaders_;
-
mutable std::mutex internal_mutex_;
-
std::shared_ptr<logging::Logger> logger_;
-
std::string name_;
};
@@ -104,28 +105,28 @@ class ProcessorFactoryWrapper : public ObjectFactoryImpl {
: ObjectFactoryImpl(factory->getGroupName()),
factory_(std::move(factory)) {}
- std::unique_ptr<CoreComponent> create(const std::string &name) override {
+ [[nodiscard]] std::unique_ptr<CoreComponent> create(const std::string &name) override {
return std::unique_ptr<CoreComponent>{createRaw(name)};
}
- std::unique_ptr<CoreComponent> create(const std::string &name, const utils::Identifier &uuid) override {
+ [[nodiscard]] std::unique_ptr<CoreComponent> create(const std::string &name, const utils::Identifier &uuid) override {
return std::unique_ptr<CoreComponent>{createRaw(name, uuid)};
}
- CoreComponent* createRaw(const std::string &name) override {
+ [[nodiscard]] CoreComponent* createRaw(const std::string &name) override {
return createRaw(name, utils::IdGenerator::getIdGenerator()->generate());
}
- CoreComponent* createRaw(const std::string &name, const utils::Identifier &uuid) override {
+ [[nodiscard]] CoreComponent* createRaw(const std::string &name, const utils::Identifier &uuid) override {
auto logger = logging::LoggerFactoryBase::getAliasedLogger(getClassName(), uuid);
return new Processor(name, uuid, factory_->create({.uuid = uuid, .name = name, .logger = logger}));
}
- std::string getGroupName() const override {
+ [[nodiscard]] std::string getGroupName() const override {
return factory_->getGroupName();
}
- std::string getClassName() override {
+ [[nodiscard]] std::string getClassName() override {
return factory_->getClassName();
}
diff --git a/minifi-api/include/minifi-cpp/core/ClassLoader.h b/minifi-api/include/minifi-cpp/core/ClassLoader.h
index ec377de61..72fa03319 100644
--- a/minifi-api/include/minifi-cpp/core/ClassLoader.h
+++ b/minifi-api/include/minifi-cpp/core/ClassLoader.h
@@ -69,13 +69,13 @@ class ClassLoader {
virtual void unregisterClass(const std::string& clazz) = 0;
- virtual std::optional<std::string> getGroupForClass(const std::string &class_name) const = 0;
+ [[nodiscard]] virtual std::optional<std::string> getGroupForClass(const std::string &class_name) const = 0;
- virtual std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) = 0;
+ [[nodiscard]] virtual std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) = 0;
- virtual std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) = 0;
+ [[nodiscard]] virtual std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) = 0;
- virtual CoreComponent* instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) = 0;
+ [[nodiscard]] virtual CoreComponent* instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) = 0;
/**
* Instantiate object based on class_name
@@ -84,7 +84,7 @@ class ClassLoader {
* @return nullptr or object created from class_name definition.
*/
template<class T = CoreComponent>
- std::unique_ptr<T> instantiate(const std::string &class_name, const std::string &name);
+ [[nodiscard]] std::unique_ptr<T> instantiate(const std::string &class_name, const std::string &name);
/**
* Instantiate object based on class_name
@@ -93,7 +93,7 @@ class ClassLoader {
* @return nullptr or object created from class_name definition.
*/
template<class T = CoreComponent>
- std::unique_ptr<T> instantiate(const std::string &class_name, const utils::Identifier &uuid);
+ [[nodiscard]] std::unique_ptr<T> instantiate(const std::string &class_name, const utils::Identifier &uuid);
/**
* Instantiate object based on class_name
@@ -102,7 +102,7 @@ class ClassLoader {
* @return nullptr or object created from class_name definition.
*/
template<class T = CoreComponent>
- T *instantiateRaw(const std::string &class_name, const std::string &name);
+ [[nodiscard]] T *instantiateRaw(const std::string &class_name, const std::string &name);
};
} // namespace org::apache::nifi::minifi::core
(nifi-minifi-cpp) 15/37: Linter fixes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 4bbe474db1619398ab74d0f17087d5d5a58d7a8c
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon May 12 15:00:40 2025 +0200
Linter fixes
---
extensions/grafana-loki/PushGrafanaLoki.h | 2 +-
.../tests/unit/ProcessorTests.cpp | 5 +++--
.../windows-event-log/ConsumeWindowsEventLog.cpp | 5 ++---
.../windows-event-log/ConsumeWindowsEventLog.h | 2 +-
libminifi/include/core/Processor.h | 2 +-
libminifi/src/core/ClassLoader.cpp | 20 +++++++-------------
libminifi/src/core/Processor.cpp | 2 +-
minifi-api/include/minifi-cpp/core/ClassLoader.h | 2 ++
8 files changed, 18 insertions(+), 22 deletions(-)
diff --git a/extensions/grafana-loki/PushGrafanaLoki.h b/extensions/grafana-loki/PushGrafanaLoki.h
index 7ca192aa1..5f8f914cb 100644
--- a/extensions/grafana-loki/PushGrafanaLoki.h
+++ b/extensions/grafana-loki/PushGrafanaLoki.h
@@ -34,7 +34,7 @@ namespace org::apache::nifi::minifi::extensions::grafana::loki {
class PushGrafanaLoki : public core::ProcessorImpl {
public:
- PushGrafanaLoki(core::ProcessorMetadata info)
+ explicit PushGrafanaLoki(core::ProcessorMetadata info)
: ProcessorImpl(info),
log_batch_(info.logger) {
}
diff --git a/extensions/standard-processors/tests/unit/ProcessorTests.cpp b/extensions/standard-processors/tests/unit/ProcessorTests.cpp
index 9bd45174c..afec8d9f2 100644
--- a/extensions/standard-processors/tests/unit/ProcessorTests.cpp
+++ b/extensions/standard-processors/tests/unit/ProcessorTests.cpp
@@ -410,8 +410,9 @@ TEST_CASE("Test Find file", "[getfileCreate3]") {
LogTestController::getInstance().setDebug<minifi::provenance::ProvenanceReporter>();
std::shared_ptr<TestPlan> plan = testController.createPlan();
auto processor = plan->addProcessor("GetFile", "getfileCreate2");
- TypedProcessorWrapper<minifi::core::reporting::SiteToSiteProvenanceReportingTask> processorReport = plan->addProcessor(minifi::test::utils::make_custom_processor<minifi::core::reporting::SiteToSiteProvenanceReportingTask>(
- std::make_shared<minifi::ConfigureImpl>()), "reporter", core::Relationship("success", "description"), false);
+ TypedProcessorWrapper<minifi::core::reporting::SiteToSiteProvenanceReportingTask> processorReport = plan->addProcessor(
+ minifi::test::utils::make_custom_processor<minifi::core::reporting::SiteToSiteProvenanceReportingTask>(std::make_shared<minifi::ConfigureImpl>()),
+ "reporter", core::Relationship("success", "description"), false);
auto dir = testController.createTempDirectory();
plan->setProperty(processor, minifi::processors::GetFile::Directory, dir.string());
diff --git a/extensions/windows-event-log/ConsumeWindowsEventLog.cpp b/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
index ec9ed90fa..391080840 100644
--- a/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
+++ b/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
@@ -61,9 +61,8 @@ namespace org::apache::nifi::minifi::processors {
const int EVT_NEXT_TIMEOUT_MS = 500;
-ConsumeWindowsEventLog::ConsumeWindowsEventLog(const std::string_view name, const utils::Identifier& uuid)
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ConsumeWindowsEventLog>::getLogger(uuid_);
+ConsumeWindowsEventLog::ConsumeWindowsEventLog(core::ProcessorMetadata info)
+ : core::ProcessorImpl(info) {
char buff[MAX_COMPUTERNAME_LENGTH + 1];
DWORD size = sizeof(buff);
if (GetComputerName(buff, &size)) {
diff --git a/extensions/windows-event-log/ConsumeWindowsEventLog.h b/extensions/windows-event-log/ConsumeWindowsEventLog.h
index 1fc66e64f..d47ab7ddf 100644
--- a/extensions/windows-event-log/ConsumeWindowsEventLog.h
+++ b/extensions/windows-event-log/ConsumeWindowsEventLog.h
@@ -80,7 +80,7 @@ class Bookmark;
class ConsumeWindowsEventLog : public core::ProcessorImpl {
public:
- explicit ConsumeWindowsEventLog(const std::string_view name, const utils::Identifier& uuid = {});
+ explicit ConsumeWindowsEventLog(core::ProcessorMetadata info);
~ConsumeWindowsEventLog() override;
diff --git a/libminifi/include/core/Processor.h b/libminifi/include/core/Processor.h
index 5558db8c5..c9503dc18 100644
--- a/libminifi/include/core/Processor.h
+++ b/libminifi/include/core/Processor.h
@@ -92,7 +92,7 @@ class Processor : public ConnectableImpl, public ConfigurableComponentImpl, publ
void setProcessGroupUUIDStr(const std::string &uuid);
void yield() override;
void yield(std::chrono::steady_clock::duration delta_time);
- bool isYield();
+ bool isYield() const;
void clearYield();
std::chrono::steady_clock::time_point getYieldExpirationTime() const;
std::chrono::steady_clock::duration getYieldTime() const;
diff --git a/libminifi/src/core/ClassLoader.cpp b/libminifi/src/core/ClassLoader.cpp
index ebbce7f27..a316887e6 100644
--- a/libminifi/src/core/ClassLoader.cpp
+++ b/libminifi/src/core/ClassLoader.cpp
@@ -25,15 +25,11 @@
#include "range/v3/action/unique.hpp"
#include "core/Processor.h"
-namespace org {
-namespace apache {
-namespace nifi {
-namespace minifi {
-namespace core {
+namespace org::apache::nifi::minifi::core {
class ClassLoaderImpl : public ClassLoader {
public:
- explicit ClassLoaderImpl(const std::string& name = "/");
+ explicit ClassLoaderImpl(std::string name = "/");
ClassLoader& getClassLoader(const std::string& child_name) override;
@@ -51,6 +47,8 @@ class ClassLoaderImpl : public ClassLoader {
CoreComponent* instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) override;
+ ~ClassLoaderImpl() override = default;
+
private:
std::map<std::string, std::unique_ptr<ObjectFactory>> loaded_factories_;
@@ -63,8 +61,8 @@ class ClassLoaderImpl : public ClassLoader {
std::string name_;
};
-ClassLoaderImpl::ClassLoaderImpl(const std::string& name)
- : logger_(logging::LoggerFactory<ClassLoader>::getLogger()), name_(name) {}
+ClassLoaderImpl::ClassLoaderImpl(std::string name)
+ : logger_(logging::LoggerFactory<ClassLoader>::getLogger()), name_(std::move(name)) {}
ClassLoader &ClassLoader::getDefaultClassLoader() {
static ClassLoaderImpl ret;
@@ -219,8 +217,4 @@ CoreComponent* ClassLoaderImpl::instantiateRaw(const std::string &class_name, co
return nullptr;
}
-} /* namespace core */
-} /* namespace minifi */
-} /* namespace nifi */
-} /* namespace apache */
-} /* namespace org */
+} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/src/core/Processor.cpp b/libminifi/src/core/Processor.cpp
index d1188c234..2ff4dc802 100644
--- a/libminifi/src/core/Processor.cpp
+++ b/libminifi/src/core/Processor.cpp
@@ -348,7 +348,7 @@ void Processor::yield(std::chrono::steady_clock::duration delta_time) {
yield_expiration_ = std::chrono::steady_clock::now() + delta_time;
}
-bool Processor::isYield() {
+bool Processor::isYield() const {
return getYieldTime() > 0ms;
}
diff --git a/minifi-api/include/minifi-cpp/core/ClassLoader.h b/minifi-api/include/minifi-cpp/core/ClassLoader.h
index cac9410d6..ec377de61 100644
--- a/minifi-api/include/minifi-cpp/core/ClassLoader.h
+++ b/minifi-api/include/minifi-cpp/core/ClassLoader.h
@@ -51,6 +51,8 @@ class ClassLoader {
public:
static ClassLoader &getDefaultClassLoader();
+ virtual ~ClassLoader() = default;
+
/**
* Retrieves a class loader
* @param name name of class loader
(nifi-minifi-cpp) 18/37: Remove obsolete file
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 34559a0fb9aae8eb5124b798c6cb0f6aceb2990a
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Jun 19 16:01:18 2025 +0200
Remove obsolete file
---
.../include/minifi-cpp/core/ProcessContextApi.h | 37 ----------------------
1 file changed, 37 deletions(-)
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContextApi.h b/minifi-api/include/minifi-cpp/core/ProcessContextApi.h
deleted file mode 100644
index 8e75ccdb5..000000000
--- a/minifi-api/include/minifi-cpp/core/ProcessContextApi.h
+++ /dev/null
@@ -1,37 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-#pragma once
-
-#include <string>
-#include <map>
-#include "utils/expected.h"
-#include "minifi-cpp/core/FlowFile.h"
-#include "minifi-cpp/core/ProcessContext.h"
-
-namespace org::apache::nifi::minifi::core {
-
-class ProcessContextApi {
- public:
- virtual nonstd::expected<std::string, std::error_code> getProperty(ProcessContext& context, std::string_view name, const FlowFile*) const = 0;
- virtual nonstd::expected<std::string, std::error_code> getDynamicProperty(ProcessContext& context, std::string_view name, const FlowFile*) const = 0;
-
- virtual std::map<std::string, std::string> getDynamicProperties(ProcessContext& context, const FlowFile*) const = 0;
-
- virtual ~ProcessContextApi() = default;
-};
-
-} // namespace org::apache::nifi::minifi::core
\ No newline at end of file
(nifi-minifi-cpp) 12/37: Remove logger
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit a57cfec50b80ea43e136ad50d9431809e3fa1893
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Apr 28 13:05:20 2025 +0200
Remove logger
---
extensions/pdh/PerformanceDataMonitor.h | 1 -
1 file changed, 1 deletion(-)
diff --git a/extensions/pdh/PerformanceDataMonitor.h b/extensions/pdh/PerformanceDataMonitor.h
index 4e52da822..08d6ddbe9 100644
--- a/extensions/pdh/PerformanceDataMonitor.h
+++ b/extensions/pdh/PerformanceDataMonitor.h
@@ -124,7 +124,6 @@ class PerformanceDataMonitor final : public core::ProcessorImpl {
bool pretty_output_;
std::optional<uint8_t> decimal_places_;
- std::shared_ptr<core::logging::Logger> logger_;
PDH_HQUERY pdh_query_;
std::vector<std::unique_ptr<PerformanceDataCounter>> resource_consumption_counters_;
};
(nifi-minifi-cpp) 04/37: Review changes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit a28b89e7afea7abcd8c3a0d37ec5d55908606557
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Wed Mar 19 09:46:17 2025 +0100
Review changes
---
extensions/aws/tests/S3TestsFixture.h | 4 +-
.../azure/processors/AzureStorageProcessorBase.h | 2 +-
.../azure/tests/AzureBlobStorageTestsFixture.h | 4 +-
.../azure/tests/AzureDataLakeStorageTestsFixture.h | 2 +-
.../azure/tests/ListAzureBlobStorageTests.cpp | 2 +-
.../azure/tests/ListAzureDataLakeStorageTests.cpp | 2 +-
extensions/libarchive/BinFiles.h | 2 +-
extensions/libarchive/CompressContent.h | 2 +-
extensions/libarchive/UnfocusArchiveEntry.h | 2 +-
extensions/mqtt/processors/ConsumeMQTT.h | 2 +-
extensions/mqtt/processors/PublishMQTT.h | 2 +-
extensions/opc/include/fetchopc.h | 1 -
extensions/opc/include/opcbase.h | 2 +-
extensions/opc/include/putopc.h | 1 -
extensions/opc/src/fetchopc.cpp | 1 -
extensions/pdh/PerformanceDataMonitor.h | 2 +-
extensions/procfs/processors/ProcFsMonitor.h | 2 +-
extensions/python/PythonProcessor.h | 2 +-
extensions/python/PythonScriptEngine.h | 2 +-
extensions/python/types/PyProcessContext.cpp | 2 +-
extensions/script/ExecuteScript.h | 2 +-
extensions/sftp/processors/FetchSFTP.h | 2 +-
extensions/sftp/processors/ListSFTP.h | 2 +-
extensions/sftp/processors/PutSFTP.h | 2 +-
extensions/smb/FetchSmb.h | 2 +-
extensions/smb/ListSmb.h | 2 +-
extensions/smb/PutSmb.h | 2 +-
.../standard-processors/modbus/FetchModbusTcp.h | 2 +-
.../processors/DefragmentText.h | 2 +-
.../standard-processors/processors/ListFile.h | 2 +-
extensions/standard-processors/processors/PutTCP.h | 3 +
extensions/standard-processors/processors/PutUDP.h | 2 +-
.../standard-processors/processors/RetryFlowFile.h | 2 +-
extensions/systemd/ConsumeJournald.h | 2 +-
extensions/test-processors/KamikazeProcessor.h | 2 +-
.../test-processors/LogOnDestructionProcessor.h | 2 +-
.../windows-event-log/ConsumeWindowsEventLog.h | 2 +-
extensions/windows-event-log/TailEventLog.h | 2 +-
extensions/windows-event-log/wel/MetadataWalker.h | 2 +-
extensions/windows-event-log/wel/WindowsEventLog.h | 2 +-
extensions/windows-event-log/wel/XMLString.h | 2 +-
libminifi/include/Port.h | 13 +-
libminifi/include/core/ProcessGroup.h | 2 +-
libminifi/include/core/Processor.h | 165 ++++++++----
libminifi/include/core/ProcessorProxy.h | 281 ---------------------
libminifi/src/core/ClassLoader.cpp | 4 +-
libminifi/src/core/FlowConfiguration.cpp | 1 -
libminifi/src/core/Processor.cpp | 189 ++++++++++++--
.../src/core/flow/StructuredConfiguration.cpp | 10 +-
libminifi/test/libtest/unit/TestUtils.h | 6 +-
.../include/minifi-cpp/core/ProcessContext.h | 1 -
.../minifi-cpp/core/ProcessContextBuilder.h | 1 -
.../core/{Processor.h => ProcessorApi.h} | 0
.../include/minifi-cpp/core/ProcessorFactory.h | 3 +-
utils/include/core/Processor.h | 19 --
utils/include/core/ProcessorImpl.h | 2 +-
.../src/core/{Processor.cpp => ProcessorImpl.cpp} | 2 +-
utils/src/core/ProcessorMetrics.cpp | 2 +-
58 files changed, 342 insertions(+), 443 deletions(-)
diff --git a/extensions/aws/tests/S3TestsFixture.h b/extensions/aws/tests/S3TestsFixture.h
index f65030f84..360e2722f 100644
--- a/extensions/aws/tests/S3TestsFixture.h
+++ b/extensions/aws/tests/S3TestsFixture.h
@@ -137,7 +137,7 @@ class FlowProcessorS3TestsFixture : public S3TestsFixture<T> {
this->mock_s3_request_sender_ptr = mock_s3_request_sender.get();
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
auto impl = std::unique_ptr<T>(new T(core::ProcessorMetadata{.uuid = uuid, .name = "S3Processor", .logger = core::logging::LoggerFactory<T>::getLogger(uuid)}, std::move(mock_s3_request_sender)));
- auto s3_processor_unique_ptr = std::make_unique<core::ProcessorProxy>("S3Processor", uuid, std::move(impl));
+ auto s3_processor_unique_ptr = std::make_unique<core::Processor>("S3Processor", uuid, std::move(impl));
this->s3_processor = s3_processor_unique_ptr.get();
auto input_dir = this->test_controller.createTempDirectory();
@@ -201,7 +201,7 @@ class FlowProducerS3TestsFixture : public S3TestsFixture<T> {
this->mock_s3_request_sender_ptr = mock_s3_request_sender.get();
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
auto impl = std::unique_ptr<T>(new T(core::ProcessorMetadata{.uuid = uuid, .name = "S3Processor", .logger = core::logging::LoggerFactory<T>::getLogger(uuid)}, std::move(mock_s3_request_sender)));
- auto s3_processor_unique_ptr = std::make_unique<core::ProcessorProxy>("S3Processor", uuid, std::move(impl));
+ auto s3_processor_unique_ptr = std::make_unique<core::Processor>("S3Processor", uuid, std::move(impl));
this->s3_processor = s3_processor_unique_ptr.get();
this->plan->addProcessor(
diff --git a/extensions/azure/processors/AzureStorageProcessorBase.h b/extensions/azure/processors/AzureStorageProcessorBase.h
index 13955ee4d..5eb0b07b1 100644
--- a/extensions/azure/processors/AzureStorageProcessorBase.h
+++ b/extensions/azure/processors/AzureStorageProcessorBase.h
@@ -29,7 +29,7 @@
#include "core/PropertyDefinition.h"
#include "core/Property.h"
#include "core/PropertyDefinitionBuilder.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/logging/Logger.h"
#include "storage/AzureStorageCredentials.h"
diff --git a/extensions/azure/tests/AzureBlobStorageTestsFixture.h b/extensions/azure/tests/AzureBlobStorageTestsFixture.h
index 68d278711..897a7905f 100644
--- a/extensions/azure/tests/AzureBlobStorageTestsFixture.h
+++ b/extensions/azure/tests/AzureBlobStorageTestsFixture.h
@@ -33,7 +33,7 @@
#include "processors/UpdateAttribute.h"
#include "utils/file/FileUtils.h"
#include "MockBlobStorage.h"
-#include "core/ProcessorProxy.h"
+#include "core/Processor.h"
const std::string CONTAINER_NAME = "test-container";
const std::string STORAGE_ACCOUNT_NAME = "test-account";
@@ -65,7 +65,7 @@ class AzureBlobStorageTestsFixture {
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
auto impl = std::unique_ptr<ProcessorType>(
new ProcessorType({.uuid = uuid, .name = "AzureBlobStorageProcessor", .logger = logging::LoggerFactory<ProcessorType>::getLogger(uuid)}, std::move(mock_blob_storage)));
- auto azure_blob_storage_processor_unique_ptr = std::make_unique<core::ProcessorProxy>(impl->getName(), impl->getUUID(), std::move(impl));
+ auto azure_blob_storage_processor_unique_ptr = std::make_unique<core::Processor>(impl->getName(), impl->getUUID(), std::move(impl));
azure_blob_storage_processor_ = azure_blob_storage_processor_unique_ptr.get();
auto input_dir = test_controller_.createTempDirectory();
std::ofstream input_file_stream(input_dir / GET_FILE_NAME);
diff --git a/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h b/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
index e49b3959a..e7f505f3c 100644
--- a/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
+++ b/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
@@ -63,7 +63,7 @@ class AzureDataLakeStorageTestsFixture {
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
auto impl = std::unique_ptr<AzureDataLakeStorageProcessor>(
new AzureDataLakeStorageProcessor({.uuid = uuid, .name = "AzureDataLakeStorageProcessor", .logger = logging::LoggerFactory<AzureDataLakeStorageProcessor>::getLogger(uuid)}, std::move(mock_data_lake_storage_client)));
- auto azure_data_lake_storage_unique_ptr = std::make_unique<core::ProcessorProxy>(impl->getName(), impl->getUUID(), std::move(impl));
+ auto azure_data_lake_storage_unique_ptr = std::make_unique<core::Processor>(impl->getName(), impl->getUUID(), std::move(impl));
azure_data_lake_storage_ = azure_data_lake_storage_unique_ptr.get();
auto input_dir = test_controller_.createTempDirectory();
minifi::test::utils::putFileToDir(input_dir, GETFILE_FILE_NAME, TEST_DATA);
diff --git a/extensions/azure/tests/ListAzureBlobStorageTests.cpp b/extensions/azure/tests/ListAzureBlobStorageTests.cpp
index a63517035..48c222d58 100644
--- a/extensions/azure/tests/ListAzureBlobStorageTests.cpp
+++ b/extensions/azure/tests/ListAzureBlobStorageTests.cpp
@@ -47,7 +47,7 @@ class ListAzureBlobStorageTestsFixture {
mock_blob_storage_ptr_ = mock_blob_storage.get();
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
auto impl = std::make_unique<minifi::azure::processors::ListAzureBlobStorage>(core::ProcessorMetadata{.uuid = uuid, .name = "ListAzureBlobStorage", .logger = logging::LoggerFactory<minifi::azure::processors::ListAzureBlobStorage>::getLogger(uuid)}, std::move(mock_blob_storage));
- auto list_azure_blob_storage_unique_ptr = std::make_unique<core::ProcessorProxy>(impl->getName(), impl->getUUID(), std::move(impl));
+ auto list_azure_blob_storage_unique_ptr = std::make_unique<core::Processor>(impl->getName(), impl->getUUID(), std::move(impl));
list_azure_blob_storage_ = list_azure_blob_storage_unique_ptr.get();
plan_->addProcessor(std::move(list_azure_blob_storage_unique_ptr), "ListAzureBlobStorage", { {"success", "d"} });
diff --git a/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp b/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
index ba89a8b5b..9335336f2 100644
--- a/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
+++ b/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
@@ -45,7 +45,7 @@ class ListAzureDataLakeStorageTestsFixture {
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
auto impl = std::unique_ptr<minifi::azure::processors::ListAzureDataLakeStorage>(
new minifi::azure::processors::ListAzureDataLakeStorage({.uuid = uuid, .name = "ListAzureDataLakeStorage", .logger = logging::LoggerFactory<minifi::azure::processors::ListAzureDataLakeStorage>::getLogger(uuid)}, std::move(mock_data_lake_storage_client)));
- auto list_azure_data_lake_storage_unique_ptr = std::make_unique<core::ProcessorProxy>(impl->getName(), impl->getUUID(), std::move(impl));
+ auto list_azure_data_lake_storage_unique_ptr = std::make_unique<core::Processor>(impl->getName(), impl->getUUID(), std::move(impl));
list_azure_data_lake_storage_ = list_azure_data_lake_storage_unique_ptr.get();
plan_->addProcessor(std::move(list_azure_data_lake_storage_unique_ptr), "ListAzureDataLakeStorage", { {"success", "d"} });
diff --git a/extensions/libarchive/BinFiles.h b/extensions/libarchive/BinFiles.h
index 5d89c1790..0c5517ee0 100644
--- a/extensions/libarchive/BinFiles.h
+++ b/extensions/libarchive/BinFiles.h
@@ -27,7 +27,7 @@
#include <utility>
#include "FlowFileRecord.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
diff --git a/extensions/libarchive/CompressContent.h b/extensions/libarchive/CompressContent.h
index 361ca871c..171a9c1e4 100644
--- a/extensions/libarchive/CompressContent.h
+++ b/extensions/libarchive/CompressContent.h
@@ -31,7 +31,7 @@
#include "archive.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/Core.h"
#include "core/PropertyDefinition.h"
diff --git a/extensions/libarchive/UnfocusArchiveEntry.h b/extensions/libarchive/UnfocusArchiveEntry.h
index f89832d5b..0261eb57f 100644
--- a/extensions/libarchive/UnfocusArchiveEntry.h
+++ b/extensions/libarchive/UnfocusArchiveEntry.h
@@ -28,7 +28,7 @@
#include "FocusArchiveEntry.h"
#include "FlowFileRecord.h"
#include "ArchiveMetadata.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/RelationshipDefinition.h"
#include "core/Core.h"
diff --git a/extensions/mqtt/processors/ConsumeMQTT.h b/extensions/mqtt/processors/ConsumeMQTT.h
index 87c2b3d31..5331763f7 100644
--- a/extensions/mqtt/processors/ConsumeMQTT.h
+++ b/extensions/mqtt/processors/ConsumeMQTT.h
@@ -26,7 +26,7 @@
#include "FlowFileRecord.h"
#include "core/Core.h"
#include "core/OutputAttributeDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/logging/LoggerFactory.h"
diff --git a/extensions/mqtt/processors/PublishMQTT.h b/extensions/mqtt/processors/PublishMQTT.h
index d6d9ed7cd..3f252e6b9 100644
--- a/extensions/mqtt/processors/PublishMQTT.h
+++ b/extensions/mqtt/processors/PublishMQTT.h
@@ -24,7 +24,7 @@
#include <vector>
#include "core/RelationshipDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/Core.h"
diff --git a/extensions/opc/include/fetchopc.h b/extensions/opc/include/fetchopc.h
index 6b7f0e04a..db23f7aa2 100644
--- a/extensions/opc/include/fetchopc.h
+++ b/extensions/opc/include/fetchopc.h
@@ -25,7 +25,6 @@
#include "opc.h"
#include "opcbase.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
#include "core/ProcessSession.h"
#include "core/Property.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/opc/include/opcbase.h b/extensions/opc/include/opcbase.h
index a8e699f20..bdb79f558 100644
--- a/extensions/opc/include/opcbase.h
+++ b/extensions/opc/include/opcbase.h
@@ -23,7 +23,7 @@
#include <vector>
#include "opc.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/opc/include/putopc.h b/extensions/opc/include/putopc.h
index 4a64b2e29..db76d4329 100644
--- a/extensions/opc/include/putopc.h
+++ b/extensions/opc/include/putopc.h
@@ -25,7 +25,6 @@
#include "opc.h"
#include "opcbase.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
#include "core/ProcessSession.h"
#include "core/Property.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/opc/src/fetchopc.cpp b/extensions/opc/src/fetchopc.cpp
index f148d2de4..7a992aa60 100644
--- a/extensions/opc/src/fetchopc.cpp
+++ b/extensions/opc/src/fetchopc.cpp
@@ -23,7 +23,6 @@
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
-#include "core/Processor.h"
#include "core/Resource.h"
#include "opc.h"
#include "utils/Enum.h"
diff --git a/extensions/pdh/PerformanceDataMonitor.h b/extensions/pdh/PerformanceDataMonitor.h
index 024d0333a..b644d81a5 100644
--- a/extensions/pdh/PerformanceDataMonitor.h
+++ b/extensions/pdh/PerformanceDataMonitor.h
@@ -25,7 +25,7 @@
#include <vector>
#include "core/logging/LoggerFactory.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/RelationshipDefinition.h"
diff --git a/extensions/procfs/processors/ProcFsMonitor.h b/extensions/procfs/processors/ProcFsMonitor.h
index 63764d0ed..c4ce0cd28 100644
--- a/extensions/procfs/processors/ProcFsMonitor.h
+++ b/extensions/procfs/processors/ProcFsMonitor.h
@@ -25,7 +25,7 @@
#include <optional>
#include "../ProcFs.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/RelationshipDefinition.h"
diff --git a/extensions/python/PythonProcessor.h b/extensions/python/PythonProcessor.h
index 84915b213..a367a7dde 100644
--- a/extensions/python/PythonProcessor.h
+++ b/extensions/python/PythonProcessor.h
@@ -23,7 +23,7 @@
#include <memory>
#include <optional>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "utils/gsl.h"
namespace org::apache::nifi::minifi::extensions::python {
diff --git a/extensions/python/PythonScriptEngine.h b/extensions/python/PythonScriptEngine.h
index e29383aaa..8b7348a9c 100644
--- a/extensions/python/PythonScriptEngine.h
+++ b/extensions/python/PythonScriptEngine.h
@@ -26,7 +26,7 @@
#include <filesystem>
#include "core/ProcessSession.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "PythonBindings.h"
#include "PyException.h"
diff --git a/extensions/python/types/PyProcessContext.cpp b/extensions/python/types/PyProcessContext.cpp
index 910e1b3f4..4d9e9af0d 100644
--- a/extensions/python/types/PyProcessContext.cpp
+++ b/extensions/python/types/PyProcessContext.cpp
@@ -21,7 +21,7 @@ a * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#include "PyStateManager.h"
#include "PyScriptFlowFile.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "controllers/RecordSetReader.h"
#include "controllers/RecordSetWriter.h"
diff --git a/extensions/script/ExecuteScript.h b/extensions/script/ExecuteScript.h
index 3cd8284ec..1a75f2c89 100644
--- a/extensions/script/ExecuteScript.h
+++ b/extensions/script/ExecuteScript.h
@@ -25,7 +25,7 @@
#include <optional>
#include "concurrentqueue.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/RelationshipDefinition.h"
diff --git a/extensions/sftp/processors/FetchSFTP.h b/extensions/sftp/processors/FetchSFTP.h
index 7a1d86f8f..510e780f7 100644
--- a/extensions/sftp/processors/FetchSFTP.h
+++ b/extensions/sftp/processors/FetchSFTP.h
@@ -23,7 +23,7 @@
#include "SFTPProcessorBase.h"
#include "utils/ByteArrayCallback.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/Core.h"
#include "core/Property.h"
diff --git a/extensions/sftp/processors/ListSFTP.h b/extensions/sftp/processors/ListSFTP.h
index daf5824ef..b8f8da831 100644
--- a/extensions/sftp/processors/ListSFTP.h
+++ b/extensions/sftp/processors/ListSFTP.h
@@ -30,7 +30,7 @@
#include <string_view>
#include "SFTPProcessorBase.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/Property.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/sftp/processors/PutSFTP.h b/extensions/sftp/processors/PutSFTP.h
index 93ecf78c9..2343bc4d7 100644
--- a/extensions/sftp/processors/PutSFTP.h
+++ b/extensions/sftp/processors/PutSFTP.h
@@ -27,7 +27,7 @@
#include "SFTPProcessorBase.h"
#include "utils/ByteArrayCallback.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/Core.h"
#include "core/Property.h"
diff --git a/extensions/smb/FetchSmb.h b/extensions/smb/FetchSmb.h
index 8f84d361d..3d3995b6d 100644
--- a/extensions/smb/FetchSmb.h
+++ b/extensions/smb/FetchSmb.h
@@ -23,7 +23,7 @@
#include <utility>
#include "SmbConnectionControllerService.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/Property.h"
#include "core/PropertyDefinition.h"
diff --git a/extensions/smb/ListSmb.h b/extensions/smb/ListSmb.h
index 3e613487e..b00e64b04 100644
--- a/extensions/smb/ListSmb.h
+++ b/extensions/smb/ListSmb.h
@@ -23,7 +23,7 @@
#include <utility>
#include "SmbConnectionControllerService.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/Property.h"
#include "core/PropertyDefinition.h"
diff --git a/extensions/smb/PutSmb.h b/extensions/smb/PutSmb.h
index 2a531a73f..a497e3b3c 100644
--- a/extensions/smb/PutSmb.h
+++ b/extensions/smb/PutSmb.h
@@ -21,7 +21,7 @@
#include <string>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "utils/Enum.h"
#include "SmbConnectionControllerService.h"
diff --git a/extensions/standard-processors/modbus/FetchModbusTcp.h b/extensions/standard-processors/modbus/FetchModbusTcp.h
index 3b21b9da0..9fb0c4d4a 100644
--- a/extensions/standard-processors/modbus/FetchModbusTcp.h
+++ b/extensions/standard-processors/modbus/FetchModbusTcp.h
@@ -18,7 +18,7 @@
#include "controllers/RecordSetWriter.h"
#include "controllers/SSLContextServiceInterface.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/logging/LoggerFactory.h"
#include "utils/net/AsioCoro.h"
diff --git a/extensions/standard-processors/processors/DefragmentText.h b/extensions/standard-processors/processors/DefragmentText.h
index 329d64d4d..300224970 100644
--- a/extensions/standard-processors/processors/DefragmentText.h
+++ b/extensions/standard-processors/processors/DefragmentText.h
@@ -23,7 +23,7 @@
#include <unordered_map>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/FlowFileStore.h"
#include "core/logging/Logger.h"
#include "core/logging/LoggerFactory.h"
diff --git a/extensions/standard-processors/processors/ListFile.h b/extensions/standard-processors/processors/ListFile.h
index 8a3037c4e..929d55e05 100644
--- a/extensions/standard-processors/processors/ListFile.h
+++ b/extensions/standard-processors/processors/ListFile.h
@@ -23,7 +23,7 @@
#include <utility>
#include "core/OutputAttributeDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
diff --git a/extensions/standard-processors/processors/PutTCP.h b/extensions/standard-processors/processors/PutTCP.h
index 71ea6858c..f95ec64c3 100644
--- a/extensions/standard-processors/processors/PutTCP.h
+++ b/extensions/standard-processors/processors/PutTCP.h
@@ -25,6 +25,9 @@
#include <utility>
#include <vector>
+#include "io/InputStream.h"
+#include "core/ProcessorImpl.h"
+#include "utils/Export.h"
#include "asio/io_context.hpp"
#include "asio/ssl/context.hpp"
#include "controllers/SSLContextServiceInterface.h"
diff --git a/extensions/standard-processors/processors/PutUDP.h b/extensions/standard-processors/processors/PutUDP.h
index e4d7bf850..b8fbbd584 100644
--- a/extensions/standard-processors/processors/PutUDP.h
+++ b/extensions/standard-processors/processors/PutUDP.h
@@ -21,7 +21,7 @@
#include <utility>
#include <vector>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/RelationshipDefinition.h"
diff --git a/extensions/standard-processors/processors/RetryFlowFile.h b/extensions/standard-processors/processors/RetryFlowFile.h
index b6db59d84..8322f82c0 100644
--- a/extensions/standard-processors/processors/RetryFlowFile.h
+++ b/extensions/standard-processors/processors/RetryFlowFile.h
@@ -27,7 +27,7 @@
#include "core/Core.h"
#include "core/OutputAttributeDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
diff --git a/extensions/systemd/ConsumeJournald.h b/extensions/systemd/ConsumeJournald.h
index 8557adb16..16a3f2e10 100644
--- a/extensions/systemd/ConsumeJournald.h
+++ b/extensions/systemd/ConsumeJournald.h
@@ -30,7 +30,7 @@
#include <vector>
#include "core/StateManager.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
diff --git a/extensions/test-processors/KamikazeProcessor.h b/extensions/test-processors/KamikazeProcessor.h
index d2a58ae9d..085fc48d9 100644
--- a/extensions/test-processors/KamikazeProcessor.h
+++ b/extensions/test-processors/KamikazeProcessor.h
@@ -19,7 +19,7 @@
#include <string>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/test-processors/LogOnDestructionProcessor.h b/extensions/test-processors/LogOnDestructionProcessor.h
index 1937dc9d6..61186865a 100644
--- a/extensions/test-processors/LogOnDestructionProcessor.h
+++ b/extensions/test-processors/LogOnDestructionProcessor.h
@@ -23,7 +23,7 @@
#include <utility>
#include "core/logging/LoggerFactory.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
namespace org::apache::nifi::minifi::processors {
diff --git a/extensions/windows-event-log/ConsumeWindowsEventLog.h b/extensions/windows-event-log/ConsumeWindowsEventLog.h
index ecb395698..1fc66e64f 100644
--- a/extensions/windows-event-log/ConsumeWindowsEventLog.h
+++ b/extensions/windows-event-log/ConsumeWindowsEventLog.h
@@ -35,7 +35,7 @@
#include <string>
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/windows-event-log/TailEventLog.h b/extensions/windows-event-log/TailEventLog.h
index d91ff85bc..e8b606f64 100644
--- a/extensions/windows-event-log/TailEventLog.h
+++ b/extensions/windows-event-log/TailEventLog.h
@@ -26,7 +26,7 @@
#include "core/Core.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/windows-event-log/wel/MetadataWalker.h b/extensions/windows-event-log/wel/MetadataWalker.h
index 869e254f1..9826f09e2 100644
--- a/extensions/windows-event-log/wel/MetadataWalker.h
+++ b/extensions/windows-event-log/wel/MetadataWalker.h
@@ -32,7 +32,7 @@
#include <utility>
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "FlowFileRecord.h"
#include "WindowsEventLog.h"
diff --git a/extensions/windows-event-log/wel/WindowsEventLog.h b/extensions/windows-event-log/wel/WindowsEventLog.h
index cef720a3b..5a2029f2d 100644
--- a/extensions/windows-event-log/wel/WindowsEventLog.h
+++ b/extensions/windows-event-log/wel/WindowsEventLog.h
@@ -32,7 +32,7 @@
#include "core/Core.h"
#include "concurrentqueue.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "utils/OsUtils.h"
#include "FlowFileRecord.h"
diff --git a/extensions/windows-event-log/wel/XMLString.h b/extensions/windows-event-log/wel/XMLString.h
index 18d35bb86..cb42218c4 100644
--- a/extensions/windows-event-log/wel/XMLString.h
+++ b/extensions/windows-event-log/wel/XMLString.h
@@ -28,7 +28,7 @@
#include <regex>
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "FlowFileRecord.h"
#include "utils/OsUtils.h"
diff --git a/libminifi/include/Port.h b/libminifi/include/Port.h
index 201d4faee..a8970e6bf 100644
--- a/libminifi/include/Port.h
+++ b/libminifi/include/Port.h
@@ -21,7 +21,7 @@
#include <utility>
#include "ForwardingNode.h"
-#include "core/ProcessorProxy.h"
+#include "core/Processor.h"
namespace org::apache::nifi::minifi {
@@ -30,10 +30,7 @@ enum class PortType {
OUTPUT
};
-class Port : public virtual core::Processor {
- public:
- virtual PortType getPortType() const = 0;
-};
+class Port;
class PortImpl final : public ForwardingNode {
public:
@@ -49,11 +46,11 @@ class PortImpl final : public ForwardingNode {
PortType port_type_;
};
-class PortProxy : public virtual Port, public core::ProcessorProxy {
+class Port : public core::Processor {
public:
- PortProxy(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<PortImpl> impl): ProcessorProxy(name, uuid, std::move(impl)) {}
+ Port(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<PortImpl> impl): Processor(name, uuid, std::move(impl)) {}
- PortType getPortType() const override {
+ PortType getPortType() const {
auto* port_impl = dynamic_cast<const PortImpl*>(impl_.get());
gsl_Assert(port_impl);
return port_impl->getPortType();
diff --git a/libminifi/include/core/ProcessGroup.h b/libminifi/include/core/ProcessGroup.h
index 1a267b04e..fc6b8ec89 100644
--- a/libminifi/include/core/ProcessGroup.h
+++ b/libminifi/include/core/ProcessGroup.h
@@ -29,7 +29,7 @@
#include <utility>
#include <tuple>
-#include "minifi-cpp/core/Processor.h"
+#include "core/Processor.h"
#include "Exception.h"
#include "TimerDrivenSchedulingAgent.h"
#include "EventDrivenSchedulingAgent.h"
diff --git a/libminifi/include/core/Processor.h b/libminifi/include/core/Processor.h
index 285b16d2d..5558db8c5 100644
--- a/libminifi/include/core/Processor.h
+++ b/libminifi/include/core/Processor.h
@@ -16,23 +16,34 @@
*/
#pragma once
+#include <algorithm>
+#include <atomic>
#include <chrono>
+#include <condition_variable>
+#include <functional>
#include <memory>
#include <mutex>
#include <string>
+#include <string_view>
#include <unordered_set>
#include <unordered_map>
+#include <utility>
+#include <vector>
-#include "minifi-cpp/core/ConfigurableComponent.h"
-#include "minifi-cpp/core/Connectable.h"
-#include "minifi-cpp/core/Property.h"
-#include "minifi-cpp/core/DynamicProperty.h"
-#include "minifi-cpp/core/Core.h"
+#include "core/ConfigurableComponentImpl.h"
+#include "core/Connectable.h"
+#include "core/Property.h"
+#include "core/Core.h"
#include "minifi-cpp/core/Annotation.h"
+#include "minifi-cpp/core/DynamicProperty.h"
#include "minifi-cpp/core/Scheduling.h"
#include "minifi-cpp/core/state/nodes/MetricsBase.h"
#include "minifi-cpp/core/ProcessorMetrics.h"
#include "utils/gsl.h"
+#include "utils/Id.h"
+#include "minifi-cpp/core/OutputAttributeDefinition.h"
+#include "Processor.h"
+#include "minifi-cpp/core/ProcessorApi.h"
namespace org::apache::nifi::minifi {
@@ -40,55 +51,78 @@ class Connection;
namespace core {
-class ProcessorApi;
class ProcessContext;
class ProcessSession;
class ProcessSessionFactory;
-class Processor : public virtual Connectable, public virtual ConfigurableComponent, public virtual state::response::ResponseNodeSource {
+class Processor : public ConnectableImpl, public ConfigurableComponentImpl, public state::response::ResponseNodeSource {
public:
- ~Processor() override = default;
-
- virtual void setScheduledState(ScheduledState state) = 0;
- virtual ScheduledState getScheduledState() const = 0;
- virtual void setSchedulingStrategy(SchedulingStrategy strategy) = 0;
- virtual SchedulingStrategy getSchedulingStrategy() const = 0;
- virtual void setSchedulingPeriod(std::chrono::steady_clock::duration period) = 0;
- virtual std::chrono::steady_clock::duration getSchedulingPeriod() const = 0;
- virtual void setCronPeriod(const std::string &period) = 0;
- virtual std::string getCronPeriod() const = 0;
- virtual void setRunDurationNano(std::chrono::steady_clock::duration period) = 0;
- virtual std::chrono::steady_clock::duration getRunDurationNano() const = 0;
- virtual void setYieldPeriodMsec(std::chrono::milliseconds period) = 0;
- virtual std::chrono::steady_clock::duration getYieldPeriod() const = 0;
- virtual void setPenalizationPeriod(std::chrono::milliseconds period) = 0;
- virtual bool isSingleThreaded() const = 0;
- virtual std::string getProcessorType() const = 0;
- virtual bool getTriggerWhenEmpty() const = 0;
- virtual uint8_t getActiveTasks() const = 0;
- virtual void incrementActiveTasks() = 0;
- virtual void decrementActiveTask() = 0;
- virtual void clearActiveTask() = 0;
- using Connectable::yield;
- virtual void yield(std::chrono::steady_clock::duration delta_time) = 0;
- virtual bool isYield() = 0;
- virtual void clearYield() = 0;
- virtual std::chrono::steady_clock::time_point getYieldExpirationTime() const = 0;
- virtual std::chrono::steady_clock::duration getYieldTime() const = 0;
- virtual bool addConnection(Connectable* connection) = 0;
- virtual void triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) = 0;
- virtual void trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) = 0;
- virtual void onTrigger(ProcessContext&, ProcessSession&) = 0;
- virtual void onSchedule(ProcessContext&, ProcessSessionFactory&) = 0;
- virtual void onUnSchedule() = 0;
- virtual bool isThrottledByBackpressure() const = 0;
- virtual void validateAnnotations() const = 0;
- virtual annotation::Input getInputRequirement() const = 0;
- virtual gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const = 0;
- virtual std::string getProcessGroupUUIDStr() const = 0;
- virtual void setProcessGroupUUIDStr(const std::string &uuid) = 0;
-
- virtual ProcessorApi& getImpl() const = 0;
+ Processor(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<ProcessorApi> impl);
+ explicit Processor(std::string_view name, std::unique_ptr<ProcessorApi> impl);
+
+ Processor(const Processor& parent) = delete;
+ Processor& operator=(const Processor& parent) = delete;
+
+ bool isRunning() const override;
+
+ ~Processor() override;
+
+ void setScheduledState(ScheduledState state);
+ ScheduledState getScheduledState() const;
+ void setSchedulingStrategy(SchedulingStrategy strategy);
+ SchedulingStrategy getSchedulingStrategy() const;
+ void setSchedulingPeriod(std::chrono::steady_clock::duration period);
+ std::chrono::steady_clock::duration getSchedulingPeriod() const;
+ void setCronPeriod(const std::string &period);
+ std::string getCronPeriod() const;
+ void setRunDurationNano(std::chrono::steady_clock::duration period);
+ std::chrono::steady_clock::duration getRunDurationNano() const;
+ void setYieldPeriodMsec(std::chrono::milliseconds period);
+ std::chrono::steady_clock::duration getYieldPeriod() const;
+ void setPenalizationPeriod(std::chrono::milliseconds period);
+ void setMaxConcurrentTasks(uint8_t tasks) override;
+ bool isSingleThreaded() const;
+ std::string getProcessorType() const;
+ bool getTriggerWhenEmpty() const;
+ uint8_t getActiveTasks() const;
+ void incrementActiveTasks();
+ void decrementActiveTask();
+ void clearActiveTask();
+ std::string getProcessGroupUUIDStr() const;
+ void setProcessGroupUUIDStr(const std::string &uuid);
+ void yield() override;
+ void yield(std::chrono::steady_clock::duration delta_time);
+ bool isYield();
+ void clearYield();
+ std::chrono::steady_clock::time_point getYieldExpirationTime() const;
+ std::chrono::steady_clock::duration getYieldTime() const;
+ bool addConnection(Connectable* connection);
+ bool canEdit() override;
+ void initialize() override;
+ void triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory);
+ void trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session);
+ void onTrigger(ProcessContext& context, ProcessSession& session);
+ void onSchedule(ProcessContext& context, ProcessSessionFactory& session_factory);
+ void onUnSchedule();
+ bool isWorkAvailable() override;
+ bool isThrottledByBackpressure() const;
+ Connectable* pickIncomingConnection() override;
+ void validateAnnotations() const;
+ annotation::Input getInputRequirement() const;
+ [[nodiscard]] bool supportsDynamicProperties() const override;
+ [[nodiscard]] bool supportsDynamicRelationships() const override;
+ state::response::SharedResponseNode getResponseNode() override;
+ gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const;
+ void restore(const std::shared_ptr<FlowFile>& file) override;
+
+ static constexpr auto DynamicProperties = std::array<DynamicProperty, 0>{};
+
+ static constexpr auto OutputAttributes = std::array<OutputAttributeReference, 0>{};
+
+ ProcessorApi& getImpl() const {
+ gsl_Assert(impl_);
+ return *impl_;
+ }
template<typename T>
T& getImpl() const {
@@ -97,8 +131,37 @@ class Processor : public virtual Connectable, public virtual ConfigurableCompone
return *res;
}
- virtual void updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force = false) = 0;
- virtual const std::unordered_map<Connection*, std::unordered_set<Processor*>>& reachable_processors() const = 0;
+ protected:
+ std::atomic<ScheduledState> state_;
+
+ std::atomic<std::chrono::steady_clock::duration> scheduling_period_;
+ std::atomic<std::chrono::steady_clock::duration> run_duration_;
+ std::atomic<std::chrono::steady_clock::duration> yield_period_;
+
+ std::atomic<uint8_t> active_tasks_;
+
+ std::string cron_period_;
+
+ std::shared_ptr<logging::Logger> logger_;
+
+ private:
+ mutable std::mutex mutex_;
+ std::atomic<std::chrono::steady_clock::time_point> yield_expiration_{};
+
+ // must hold the graphMutex
+ void updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force = false);
+
+ const std::unordered_map<Connection*, std::unordered_set<Processor*>>& reachable_processors() const;
+
+ static bool partOfCycle(Connection* conn);
+
+ // an outgoing connection allows us to reach these nodes
+ std::unordered_map<Connection*, std::unordered_set<Processor*>> reachable_processors_;
+
+ std::string process_group_uuid_;
+
+ protected:
+ std::unique_ptr<ProcessorApi> impl_;
};
} // namespace core
diff --git a/libminifi/include/core/ProcessorProxy.h b/libminifi/include/core/ProcessorProxy.h
deleted file mode 100644
index 0ad48bb3e..000000000
--- a/libminifi/include/core/ProcessorProxy.h
+++ /dev/null
@@ -1,281 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-#pragma once
-
-#include <algorithm>
-#include <atomic>
-#include <chrono>
-#include <condition_variable>
-#include <functional>
-#include <memory>
-#include <mutex>
-#include <string>
-#include <string_view>
-#include <unordered_set>
-#include <unordered_map>
-#include <utility>
-#include <vector>
-
-#include "core/ConfigurableComponentImpl.h"
-#include "core/Connectable.h"
-#include "core/Property.h"
-#include "core/Core.h"
-#include "minifi-cpp/core/Annotation.h"
-#include "minifi-cpp/core/DynamicProperty.h"
-#include "minifi-cpp/core/Scheduling.h"
-#include "minifi-cpp/core/state/nodes/MetricsBase.h"
-#include "minifi-cpp/core/ProcessorMetrics.h"
-#include "utils/gsl.h"
-#include "utils/Id.h"
-#include "minifi-cpp/core/OutputAttributeDefinition.h"
-#include "Processor.h"
-#include "minifi-cpp/core/Processor.h"
-
-namespace org::apache::nifi::minifi {
-
-class Connection;
-
-namespace core {
-
-class ProcessContext;
-class ProcessSession;
-class ProcessSessionFactory;
-
-constexpr std::chrono::microseconds MINIMUM_SCHEDULING_PERIOD{30};
-
-#define BUILDING_DLL 1
-
-class ProcessorProxy : public virtual Processor, public ConnectableImpl, public ConfigurableComponentImpl {
- public:
- ProcessorProxy(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<ProcessorApi> impl);
- explicit ProcessorProxy(std::string_view name, std::unique_ptr<ProcessorApi> impl);
-
- ProcessorProxy(const ProcessorProxy& parent) = delete;
- ProcessorProxy& operator=(const ProcessorProxy& parent) = delete;
-
- bool isRunning() const override;
-
- ~ProcessorProxy() override;
-
- void setScheduledState(ScheduledState state) override;
-
- ScheduledState getScheduledState() const override {
- return state_;
- }
-
- void setSchedulingStrategy(SchedulingStrategy strategy) override {
- strategy_ = strategy;
- }
-
- SchedulingStrategy getSchedulingStrategy() const override {
- return strategy_;
- }
-
- void setSchedulingPeriod(std::chrono::steady_clock::duration period) override {
- scheduling_period_ = std::max(std::chrono::steady_clock::duration(MINIMUM_SCHEDULING_PERIOD), period);
- }
-
- std::chrono::steady_clock::duration getSchedulingPeriod() const override {
- return scheduling_period_;
- }
-
- void setCronPeriod(const std::string &period) override {
- cron_period_ = period;
- }
-
- std::string getCronPeriod() const override {
- return cron_period_;
- }
-
- void setRunDurationNano(std::chrono::steady_clock::duration period) override {
- run_duration_ = period;
- }
-
- std::chrono::steady_clock::duration getRunDurationNano() const override {
- return (run_duration_);
- }
-
- void setYieldPeriodMsec(std::chrono::milliseconds period) override {
- yield_period_ = period;
- }
-
- std::chrono::steady_clock::duration getYieldPeriod() const override {
- return yield_period_;
- }
-
- void setPenalizationPeriod(std::chrono::milliseconds period) override {
- penalization_period_ = period;
- }
-
- void setMaxConcurrentTasks(uint8_t tasks) override;
-
- bool isSingleThreaded() const override {
- return impl_->isSingleThreaded();
- }
-
- std::string getProcessorType() const override {
- return impl_->getProcessorType();
- }
-
- bool getTriggerWhenEmpty() const override {
- return impl_->getTriggerWhenEmpty();
- }
-
- uint8_t getActiveTasks() const override {
- return (active_tasks_);
- }
-
- void incrementActiveTasks() override {
- ++active_tasks_;
- }
-
- void decrementActiveTask() override {
- if (active_tasks_ > 0)
- --active_tasks_;
- }
-
- void clearActiveTask() override {
- active_tasks_ = 0;
- }
-
- std::string getProcessGroupUUIDStr() const override {
- return process_group_uuid_;
- }
-
- void setProcessGroupUUIDStr(const std::string &uuid) override {
- process_group_uuid_ = uuid;
- }
-
- void yield() override;
-
- void yield(std::chrono::steady_clock::duration delta_time) override;
-
- bool isYield() override;
-
- void clearYield() override;
-
- std::chrono::steady_clock::time_point getYieldExpirationTime() const override { return yield_expiration_; }
- std::chrono::steady_clock::duration getYieldTime() const override;
-
- bool addConnection(Connectable* connection) override;
-
- bool canEdit() override {
- return !isRunning();
- }
-
- void initialize() override;
-
- void triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) override;
- void trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) override;
-
- void onTrigger(ProcessContext& context, ProcessSession& session) override {
- impl_->onTrigger(context, session);
- }
-
- void onSchedule(ProcessContext& context, ProcessSessionFactory& session_factory) override {
- impl_->onSchedule(context, session_factory);
- }
-
- // Hook executed when onSchedule fails (throws). Configuration should be reset in this
- void onUnSchedule() override {
- impl_->onUnSchedule();
- }
-
- // Check all incoming connections for work
- bool isWorkAvailable() override;
-
- bool isThrottledByBackpressure() const override;
-
- Connectable* pickIncomingConnection() override;
-
- void validateAnnotations() const override;
-
- annotation::Input getInputRequirement() const override {
- return impl_->getInputRequirement();
- }
-
- [[nodiscard]] bool supportsDynamicProperties() const override {
- return impl_->supportsDynamicProperties();
- }
-
- [[nodiscard]] bool supportsDynamicRelationships() const override {
- return impl_->supportsDynamicRelationships();
- }
-
- state::response::SharedResponseNode getResponseNode() override {
- return getMetrics();
- }
-
- gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const override {
- return impl_->getMetrics();
- }
-
- ProcessorApi& getImpl() const override {
- gsl_Assert(impl_);
- return *impl_;
- }
-
- void restore(const std::shared_ptr<FlowFile>& file) override {
- impl_->restore(file);
- }
-
- static constexpr auto DynamicProperties = std::array<DynamicProperty, 0>{};
-
- static constexpr auto OutputAttributes = std::array<OutputAttributeReference, 0>{};
-
- protected:
- std::atomic<ScheduledState> state_;
-
- std::atomic<std::chrono::steady_clock::duration> scheduling_period_;
- std::atomic<std::chrono::steady_clock::duration> run_duration_;
- std::atomic<std::chrono::steady_clock::duration> yield_period_;
-
- std::atomic<uint8_t> active_tasks_;
-
- std::string cron_period_;
-
- std::shared_ptr<logging::Logger> logger_;
-
- private:
- mutable std::mutex mutex_;
- std::atomic<std::chrono::steady_clock::time_point> yield_expiration_{};
-
- static std::mutex& getGraphMutex() {
- static std::mutex mutex{};
- return mutex;
- }
-
- // must hold the graphMutex
- void updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force = false) override;
-
- const std::unordered_map<Connection*, std::unordered_set<Processor*>>& reachable_processors() const override {
- return reachable_processors_;
- }
-
- static bool partOfCycle(Connection* conn);
-
- // an outgoing connection allows us to reach these nodes
- std::unordered_map<Connection*, std::unordered_set<Processor*>> reachable_processors_;
-
- std::string process_group_uuid_;
-
- protected:
- std::unique_ptr<ProcessorApi> impl_;
-};
-
-} // namespace core
-} // namespace org::apache::nifi::minifi
diff --git a/libminifi/src/core/ClassLoader.cpp b/libminifi/src/core/ClassLoader.cpp
index 9d56c36e5..aa4be367c 100644
--- a/libminifi/src/core/ClassLoader.cpp
+++ b/libminifi/src/core/ClassLoader.cpp
@@ -23,7 +23,7 @@
#include "core/logging/LoggerFactory.h"
#include "range/v3/action/sort.hpp"
#include "range/v3/action/unique.hpp"
-#include "core/ProcessorProxy.h"
+#include "core/Processor.h"
namespace org {
namespace apache {
@@ -120,7 +120,7 @@ class ProcessorFactoryWrapper : public ObjectFactoryImpl {
CoreComponent* createRaw(const std::string &name, const utils::Identifier &uuid) override {
auto logger = logging::LoggerFactoryBase::getAliasedLogger(getClassName(), uuid);
- return new ProcessorProxy(name, uuid, factory_->create({.uuid = uuid, .name = name, .logger = logger}));
+ return new Processor(name, uuid, factory_->create({.uuid = uuid, .name = name, .logger = logger}));
}
std::string getGroupName() const override {
diff --git a/libminifi/src/core/FlowConfiguration.cpp b/libminifi/src/core/FlowConfiguration.cpp
index 8c118a635..2e1659dc8 100644
--- a/libminifi/src/core/FlowConfiguration.cpp
+++ b/libminifi/src/core/FlowConfiguration.cpp
@@ -26,7 +26,6 @@
#include "processors/ProcessorUtils.h"
#include "utils/StringUtils.h"
#include "utils/file/FileUtils.h"
-#include "core/ProcessorProxy.h"
namespace org::apache::nifi::minifi::core {
diff --git a/libminifi/src/core/Processor.cpp b/libminifi/src/core/Processor.cpp
index c195b3191..a70867ce7 100644
--- a/libminifi/src/core/Processor.cpp
+++ b/libminifi/src/core/Processor.cpp
@@ -38,14 +38,21 @@
#include "range/v3/algorithm/any_of.hpp"
#include "fmt/format.h"
#include "Exception.h"
-#include "core/ProcessorProxy.h"
+#include "core/Processor.h"
#include "core/ProcessorMetrics.h"
using namespace std::literals::chrono_literals;
namespace org::apache::nifi::minifi::core {
-ProcessorProxy::ProcessorProxy(std::string_view name, std::unique_ptr<ProcessorApi> impl)
+constexpr std::chrono::microseconds MINIMUM_SCHEDULING_PERIOD{30};
+
+static std::mutex& getGraphMutex() {
+ static std::mutex mutex{};
+ return mutex;
+}
+
+Processor::Processor(std::string_view name, std::unique_ptr<ProcessorApi> impl)
: ConnectableImpl(name),
state_(DISABLED),
scheduling_period_(MINIMUM_SCHEDULING_PERIOD),
@@ -63,7 +70,7 @@ ProcessorProxy::ProcessorProxy(std::string_view name, std::unique_ptr<ProcessorA
logger_->log_debug("Processor {} created UUID {}", name_, getUUIDStr());
}
-ProcessorProxy::ProcessorProxy(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<ProcessorApi> impl)
+Processor::Processor(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<ProcessorApi> impl)
: ConnectableImpl(name, uuid),
state_(DISABLED),
scheduling_period_(MINIMUM_SCHEDULING_PERIOD),
@@ -81,22 +88,22 @@ ProcessorProxy::ProcessorProxy(std::string_view name, const utils::Identifier& u
logger_->log_debug("Processor {} created with uuid {}", name_, getUUIDStr());
}
-ProcessorProxy::~ProcessorProxy() {
+Processor::~Processor() {
logger_->log_debug("Destroying processor {} with uuid {}", name_, getUUIDStr());
}
-bool ProcessorProxy::isRunning() const {
+bool Processor::isRunning() const {
return (state_ == RUNNING && active_tasks_ > 0);
}
-void ProcessorProxy::setScheduledState(ScheduledState state) {
+void Processor::setScheduledState(ScheduledState state) {
state_ = state;
if (state == STOPPED) {
impl_->notifyStop();
}
}
-bool ProcessorProxy::addConnection(Connectable* conn) {
+bool Processor::addConnection(Connectable* conn) {
enum class SetAs{
NONE,
OUTPUT,
@@ -166,7 +173,7 @@ bool ProcessorProxy::addConnection(Connectable* conn) {
return result != SetAs::NONE;
}
-void ProcessorProxy::triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) {
+void Processor::triggerAndCommit(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSessionFactory>& session_factory) {
const auto process_session = session_factory->createSession();
process_session->setMetrics(getMetrics());
try {
@@ -184,14 +191,14 @@ void ProcessorProxy::triggerAndCommit(const std::shared_ptr<ProcessContext>& con
}
}
-void ProcessorProxy::trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) {
+void Processor::trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) {
++impl_->getMetrics()->invocations();
const auto start = std::chrono::steady_clock::now();
onTrigger(*context, *process_session);
impl_->getMetrics()->addLastOnTriggerRuntime(std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - start));
}
-bool ProcessorProxy::isWorkAvailable() {
+bool Processor::isWorkAvailable() {
// We have work if any incoming connection has work
std::lock_guard<std::mutex> lock(mutex_);
bool hasWork = false;
@@ -217,7 +224,7 @@ bool ProcessorProxy::isWorkAvailable() {
}
// must hold the graphMutex
-void ProcessorProxy::updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force) {
+void Processor::updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force) {
bool didChange = force;
for (auto& outIt : outgoing_connections_) {
for (auto& outConn : outIt.second) {
@@ -257,7 +264,7 @@ void ProcessorProxy::updateReachability(const std::lock_guard<std::mutex>& graph
}
}
-bool ProcessorProxy::partOfCycle(Connection* conn) {
+bool Processor::partOfCycle(Connection* conn) {
auto source = dynamic_cast<Processor*>(conn->getSource());
if (!source) {
return false;
@@ -269,7 +276,7 @@ bool ProcessorProxy::partOfCycle(Connection* conn) {
return it->second.contains(source);
}
-bool ProcessorProxy::isThrottledByBackpressure() const {
+bool Processor::isThrottledByBackpressure() const {
bool isThrottledByOutgoing = ranges::any_of(outgoing_connections_, [](auto& name_connection_set_pair) {
return ranges::any_of(name_connection_set_pair.second, [](auto& connectable) {
auto connection = dynamic_cast<Connection*>(connectable);
@@ -283,7 +290,7 @@ bool ProcessorProxy::isThrottledByBackpressure() const {
return isThrottledByOutgoing && !isForcedByIncomingCycle;
}
-Connectable* ProcessorProxy::pickIncomingConnection() {
+Connectable* Processor::pickIncomingConnection() {
std::lock_guard<std::mutex> rel_guard(relationship_mutex_);
auto beginIt = incoming_connections_Iter;
@@ -303,7 +310,7 @@ Connectable* ProcessorProxy::pickIncomingConnection() {
return getNextIncomingConnectionImpl(rel_guard);
}
-void ProcessorProxy::validateAnnotations() const {
+void Processor::validateAnnotations() const {
switch (getInputRequirement()) {
case annotation::Input::INPUT_REQUIRED: {
if (!hasIncomingConnections()) {
@@ -323,7 +330,7 @@ void ProcessorProxy::validateAnnotations() const {
}
}
-void ProcessorProxy::setMaxConcurrentTasks(const uint8_t tasks) {
+void Processor::setMaxConcurrentTasks(const uint8_t tasks) {
if (isSingleThreaded() && tasks > 1) {
logger_->log_warn("Processor {} can not be run in parallel, its \"max concurrent tasks\" value is too high. "
"It was set to 1 from {}.", name_, tasks);
@@ -334,23 +341,23 @@ void ProcessorProxy::setMaxConcurrentTasks(const uint8_t tasks) {
max_concurrent_tasks_ = tasks;
}
-void ProcessorProxy::yield() {
+void Processor::yield() {
yield_expiration_ = std::chrono::steady_clock::now() + yield_period_.load();
}
-void ProcessorProxy::yield(std::chrono::steady_clock::duration delta_time) {
+void Processor::yield(std::chrono::steady_clock::duration delta_time) {
yield_expiration_ = std::chrono::steady_clock::now() + delta_time;
}
-bool ProcessorProxy::isYield() {
+bool Processor::isYield() {
return getYieldTime() > 0ms;
}
-void ProcessorProxy::clearYield() {
+void Processor::clearYield() {
yield_expiration_ = std::chrono::steady_clock::time_point();
}
-std::chrono::steady_clock::duration ProcessorProxy::getYieldTime() const {
+std::chrono::steady_clock::duration Processor::getYieldTime() const {
return std::max(yield_expiration_.load()-std::chrono::steady_clock::now(), std::chrono::steady_clock::duration{0});
}
@@ -358,7 +365,7 @@ namespace {
class ProcessorDescriptorImpl : public ProcessorDescriptor {
public:
- explicit ProcessorDescriptorImpl(ProcessorProxy* impl): impl_(impl) {}
+ explicit ProcessorDescriptorImpl(Processor* impl): impl_(impl) {}
void setSupportedRelationships(std::span<const RelationshipDefinition> relationships) override {
impl_->setSupportedRelationships(relationships);
}
@@ -368,14 +375,148 @@ class ProcessorDescriptorImpl : public ProcessorDescriptor {
}
private:
- ProcessorProxy* impl_;
+ Processor* impl_;
};
} // namespace
-void ProcessorProxy::initialize() {
+void Processor::initialize() {
ProcessorDescriptorImpl self{this};
impl_->initialize(self);
}
+ScheduledState Processor::getScheduledState() const {
+ return state_;
+}
+
+void Processor::setSchedulingStrategy(SchedulingStrategy strategy) {
+ strategy_ = strategy;
+}
+
+SchedulingStrategy Processor::getSchedulingStrategy() const {
+ return strategy_;
+}
+
+void Processor::setSchedulingPeriod(std::chrono::steady_clock::duration period) {
+ scheduling_period_ = std::max(std::chrono::steady_clock::duration(MINIMUM_SCHEDULING_PERIOD), period);
+}
+
+std::chrono::steady_clock::duration Processor::getSchedulingPeriod() const {
+ return scheduling_period_;
+}
+
+void Processor::setCronPeriod(const std::string &period) {
+ cron_period_ = period;
+}
+
+std::string Processor::getCronPeriod() const {
+ return cron_period_;
+}
+
+void Processor::setRunDurationNano(std::chrono::steady_clock::duration period) {
+ run_duration_ = period;
+}
+
+std::chrono::steady_clock::duration Processor::getRunDurationNano() const {
+ return (run_duration_);
+}
+
+void Processor::setYieldPeriodMsec(std::chrono::milliseconds period) {
+ yield_period_ = period;
+}
+
+std::chrono::steady_clock::duration Processor::getYieldPeriod() const {
+ return yield_period_;
+}
+
+void Processor::setPenalizationPeriod(std::chrono::milliseconds period) {
+ penalization_period_ = period;
+}
+
+bool Processor::isSingleThreaded() const {
+ return impl_->isSingleThreaded();
+}
+
+std::string Processor::getProcessorType() const {
+ return impl_->getProcessorType();
+}
+
+bool Processor::getTriggerWhenEmpty() const {
+ return impl_->getTriggerWhenEmpty();
+}
+
+uint8_t Processor::getActiveTasks() const {
+ return (active_tasks_);
+}
+
+void Processor::incrementActiveTasks() {
+ ++active_tasks_;
+}
+
+void Processor::decrementActiveTask() {
+ if (active_tasks_ > 0)
+ --active_tasks_;
+}
+
+void Processor::clearActiveTask() {
+ active_tasks_ = 0;
+}
+
+std::string Processor::getProcessGroupUUIDStr() const {
+ return process_group_uuid_;
+}
+
+void Processor::setProcessGroupUUIDStr(const std::string &uuid) {
+ process_group_uuid_ = uuid;
+}
+
+std::chrono::steady_clock::time_point Processor::getYieldExpirationTime() const {
+ return yield_expiration_;
+}
+
+bool Processor::canEdit() {
+ return !isRunning();
+}
+
+void Processor::onTrigger(ProcessContext& context, ProcessSession& session) {
+ impl_->onTrigger(context, session);
+}
+
+void Processor::onSchedule(ProcessContext& context, ProcessSessionFactory& session_factory) {
+ impl_->onSchedule(context, session_factory);
+}
+
+// Hook executed when onSchedule fails (throws). Configuration should be reset in this
+void Processor::onUnSchedule() {
+ impl_->onUnSchedule();
+}
+
+annotation::Input Processor::getInputRequirement() const {
+ return impl_->getInputRequirement();
+}
+
+[[nodiscard]] bool Processor::supportsDynamicProperties() const {
+ return impl_->supportsDynamicProperties();
+}
+
+[[nodiscard]] bool Processor::supportsDynamicRelationships() const {
+ return impl_->supportsDynamicRelationships();
+}
+
+state::response::SharedResponseNode Processor::getResponseNode() {
+ return getMetrics();
+}
+
+gsl::not_null<std::shared_ptr<ProcessorMetrics>> Processor::getMetrics() const {
+ return impl_->getMetrics();
+}
+
+void Processor::restore(const std::shared_ptr<FlowFile>& file) {
+ impl_->restore(file);
+}
+
+const std::unordered_map<Connection*, std::unordered_set<Processor*>>& Processor::reachable_processors() const {
+ return reachable_processors_;
+}
+
} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/src/core/flow/StructuredConfiguration.cpp b/libminifi/src/core/flow/StructuredConfiguration.cpp
index 61f8d8582..667aa3205 100644
--- a/libminifi/src/core/flow/StructuredConfiguration.cpp
+++ b/libminifi/src/core/flow/StructuredConfiguration.cpp
@@ -24,7 +24,7 @@
#include "Funnel.h"
#include "core/ParameterContext.h"
-#include "core/ProcessorProxy.h"
+#include "core/Processor.h"
#include "core/ParameterTokenParser.h"
#include "core/ReferenceParser.h"
#include "core/flow/CheckRequiredField.h"
@@ -584,7 +584,7 @@ void StructuredConfiguration::parseProvenanceReporting(const Node& node, core::P
auto report_task_impl = createProvenanceReportTask();
auto* reportTask = report_task_impl.get();
- auto report_task_wrapper = std::make_unique<core::ProcessorProxy>("", std::move(report_task_impl));
+ auto report_task_wrapper = std::make_unique<core::Processor>("", std::move(report_task_impl));
checkRequiredField(node, schema_.scheduling_strategy);
auto schedulingStrategyStr = node[schema_.scheduling_strategy].getString().value();
@@ -747,7 +747,7 @@ void StructuredConfiguration::parseRPGPort(const Node& port_node, core::ProcessG
auto port_impl = std::make_unique<minifi::RemoteProcessorGroupPort>(
nameStr, parent->getURL(), this->configuration_, uuid);
auto* port = port_impl.get();
- auto port_wrapper = std::make_unique<core::ProcessorProxy>(nameStr, uuid, std::move(port_impl));
+ auto port_wrapper = std::make_unique<core::Processor>(nameStr, uuid, std::move(port_impl));
port->setDirection(direction);
port->setTimeout(parent->getTimeout());
port->setTransmitting(true);
@@ -952,7 +952,7 @@ void StructuredConfiguration::parseFunnels(const Node& node, core::ProcessGroup*
throw Exception(ExceptionType::GENERAL_EXCEPTION, "Incorrect funnel UUID format.");
});
- auto funnel = std::make_unique<core::ProcessorProxy>(name, uuid.value(), std::make_unique<minifi::Funnel>(name, uuid.value()));
+ auto funnel = std::make_unique<core::Processor>(name, uuid.value(), std::make_unique<minifi::Funnel>(name, uuid.value()));
logger_->log_debug("Created funnel with UUID {} and name {}", id, name);
funnel->setScheduledState(core::RUNNING);
funnel->setSchedulingStrategy(core::EVENT_DRIVEN);
@@ -980,7 +980,7 @@ void StructuredConfiguration::parsePorts(const flow::Node& node, core::ProcessGr
throw Exception(ExceptionType::GENERAL_EXCEPTION, "Incorrect port UUID format.");
});
- auto port = std::make_unique<PortProxy>(name, uuid.value(), std::make_unique<PortImpl>(name, uuid.value(), port_type));
+ auto port = std::make_unique<Port>(name, uuid.value(), std::make_unique<PortImpl>(name, uuid.value(), port_type));
logger_->log_debug("Created port UUID {} and name {}", id, name);
port->setScheduledState(core::RUNNING);
port->setSchedulingStrategy(core::EVENT_DRIVEN);
diff --git a/libminifi/test/libtest/unit/TestUtils.h b/libminifi/test/libtest/unit/TestUtils.h
index 2c2ef7bcb..ff3b435a3 100644
--- a/libminifi/test/libtest/unit/TestUtils.h
+++ b/libminifi/test/libtest/unit/TestUtils.h
@@ -39,7 +39,7 @@
#include "asio/ssl.hpp"
#include "utils/net/Ssl.h"
#include "range/v3/algorithm/any_of.hpp"
-#include "core/ProcessorProxy.h"
+#include "core/Processor.h"
#include "core/logging/LoggerFactory.h"
using namespace std::literals::chrono_literals;
@@ -243,7 +243,7 @@ std::unique_ptr<core::Processor> make_processor(std::string_view name, std::opti
.name = std::string{name},
.logger = minifi::core::logging::LoggerFactory<T>::getLogger(uuid.value())
});
- return std::make_unique<core::ProcessorProxy>(name, uuid.value(), std::move(processor_impl));
+ return std::make_unique<core::Processor>(name, uuid.value(), std::move(processor_impl));
}
template<typename T, typename ...Args>
@@ -251,7 +251,7 @@ std::unique_ptr<core::Processor> make_custom_processor(Args&&... args) {
auto processor_impl = std::make_unique<T>(std::forward<Args>(args)...);
auto name = processor_impl->getName();
auto uuid = processor_impl->getUUID();
- return std::make_unique<core::ProcessorProxy>(name, uuid, std::move(processor_impl));
+ return std::make_unique<core::Processor>(name, uuid, std::move(processor_impl));
}
} // namespace org::apache::nifi::minifi::test::utils
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContext.h b/minifi-api/include/minifi-cpp/core/ProcessContext.h
index d925d9f55..629d06c28 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContext.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContext.h
@@ -24,7 +24,6 @@
#include "minifi-cpp/core/Core.h"
#include "minifi-cpp/core/ContentRepository.h"
#include "minifi-cpp/core/controller/ControllerServiceLookup.h"
-#include "minifi-cpp/core/Processor.h"
#include "minifi-cpp/core/Property.h"
#include "minifi-cpp/core/Repository.h"
#include "minifi-cpp/core/FlowFile.h"
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContextBuilder.h b/minifi-api/include/minifi-cpp/core/ProcessContextBuilder.h
index 18d8ec27f..7607dfee2 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContextBuilder.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContextBuilder.h
@@ -22,7 +22,6 @@
#include "minifi-cpp/properties/Configure.h"
#include "minifi-cpp/core/controller/ControllerServiceProvider.h"
#include "ProcessContext.h"
-#include "Processor.h"
#include "minifi-cpp/core/Repository.h"
namespace org::apache::nifi::minifi::core {
diff --git a/minifi-api/include/minifi-cpp/core/Processor.h b/minifi-api/include/minifi-cpp/core/ProcessorApi.h
similarity index 100%
rename from minifi-api/include/minifi-cpp/core/Processor.h
rename to minifi-api/include/minifi-cpp/core/ProcessorApi.h
diff --git a/minifi-api/include/minifi-cpp/core/ProcessorFactory.h b/minifi-api/include/minifi-cpp/core/ProcessorFactory.h
index 7b453170f..ed27ebf2e 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessorFactory.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessorFactory.h
@@ -20,11 +20,12 @@
#include <string>
#include <memory>
#include <utility>
-#include "minifi-cpp/core/Processor.h"
#include "minifi-cpp/core/ProcessorMetadata.h"
namespace org::apache::nifi::minifi::core {
+class ProcessorApi;
+
class ProcessorFactory {
public:
virtual std::unique_ptr<ProcessorApi> create(ProcessorMetadata info) = 0;
diff --git a/utils/include/core/Processor.h b/utils/include/core/Processor.h
deleted file mode 100644
index 3792098b7..000000000
--- a/utils/include/core/Processor.h
+++ /dev/null
@@ -1,19 +0,0 @@
-/**
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-#pragma once
-
-#include "core/ProcessorImpl.h"
diff --git a/utils/include/core/ProcessorImpl.h b/utils/include/core/ProcessorImpl.h
index 225e2b243..23ba2f516 100644
--- a/utils/include/core/ProcessorImpl.h
+++ b/utils/include/core/ProcessorImpl.h
@@ -42,7 +42,7 @@
#include "utils/gsl.h"
#include "utils/Id.h"
#include "minifi-cpp/core/OutputAttributeDefinition.h"
-#include "minifi-cpp/core/Processor.h"
+#include "minifi-cpp/core/ProcessorApi.h"
#include "utils/PropertyErrors.h"
#include "minifi-cpp/core/ProcessorMetadata.h"
#include "Exception.h"
diff --git a/utils/src/core/Processor.cpp b/utils/src/core/ProcessorImpl.cpp
similarity index 98%
rename from utils/src/core/Processor.cpp
rename to utils/src/core/ProcessorImpl.cpp
index 0caaf9c67..460ba47ff 100644
--- a/utils/src/core/Processor.cpp
+++ b/utils/src/core/ProcessorImpl.cpp
@@ -17,7 +17,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include <ctime>
#include <cctype>
diff --git a/utils/src/core/ProcessorMetrics.cpp b/utils/src/core/ProcessorMetrics.cpp
index 5d3e7be33..9952f74b4 100644
--- a/utils/src/core/ProcessorMetrics.cpp
+++ b/utils/src/core/ProcessorMetrics.cpp
@@ -16,7 +16,7 @@
*/
#include "core/ProcessorMetrics.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/state/Value.h"
#include "utils/gsl.h"
#include "range/v3/numeric/accumulate.hpp"
(nifi-minifi-cpp) 31/37: Review changes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit b43f26bc2902ef3d47a114d604a5fdbb169c4960
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Jul 17 14:12:33 2025 +0200
Review changes
---
.../{ProcessorFactory.h => ProcessorFactoryImpl.h} | 4 ++--
core-framework/include/core/Resource.h | 2 +-
libminifi/include/core/FlowConfiguration.h | 2 +-
.../reporting/SiteToSiteProvenanceReportingTask.h | 2 +-
libminifi/src/core/FlowConfiguration.cpp | 6 ++++--
.../src/core/flow/StructuredConfiguration.cpp | 22 +++++++++-------------
6 files changed, 18 insertions(+), 20 deletions(-)
diff --git a/core-framework/include/core/ProcessorFactory.h b/core-framework/include/core/ProcessorFactoryImpl.h
similarity index 95%
rename from core-framework/include/core/ProcessorFactory.h
rename to core-framework/include/core/ProcessorFactoryImpl.h
index bbba22664..d4ea016c8 100644
--- a/core-framework/include/core/ProcessorFactory.h
+++ b/core-framework/include/core/ProcessorFactoryImpl.h
@@ -46,12 +46,12 @@ class ProcessorFactoryImpl : public ProcessorFactory {
}
std::string getClassName() const override {
- return class_name_;
+ return std::string{class_name_};
}
protected:
std::string group_name_;
- std::string class_name_;
+ std::string_view class_name_;
};
} // namespace org::apache::nifi::minifi::core
diff --git a/core-framework/include/core/Resource.h b/core-framework/include/core/Resource.h
index c4994666a..999f1ddb5 100644
--- a/core-framework/include/core/Resource.h
+++ b/core-framework/include/core/Resource.h
@@ -29,7 +29,7 @@
#include "agent/agent_docs.h"
#include "utils/OptionalUtils.h"
#include "utils/Macro.h"
-#include "core/ProcessorFactory.h"
+#include "core/ProcessorFactoryImpl.h"
namespace org::apache::nifi::minifi::core {
diff --git a/libminifi/include/core/FlowConfiguration.h b/libminifi/include/core/FlowConfiguration.h
index 8d83efcaf..0c9cd327c 100644
--- a/libminifi/include/core/FlowConfiguration.h
+++ b/libminifi/include/core/FlowConfiguration.h
@@ -96,7 +96,7 @@ class FlowConfiguration : public CoreComponentImpl {
// Create Connection
[[nodiscard]] std::unique_ptr<minifi::Connection> createConnection(const std::string &name, const utils::Identifier &uuid) const;
// Create Provenance Report Task
- std::unique_ptr<core::reporting::SiteToSiteProvenanceReportingTask> createProvenanceReportTask();
+ std::unique_ptr<core::Processor> createProvenanceReportTask();
static std::unique_ptr<core::ParameterProvider> createParameterProvider(const std::string &class_name, const std::string &full_class_name, const utils::Identifier& uuid);
diff --git a/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h b/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
index 6998e922d..dea93e616 100644
--- a/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
+++ b/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
@@ -36,7 +36,7 @@ class SiteToSiteProvenanceReportingTask : public minifi::RemoteProcessorGroupPor
public:
explicit SiteToSiteProvenanceReportingTask(std::shared_ptr<Configure> configure)
: minifi::RemoteProcessorGroupPort(ReportTaskName, "", std::move(configure),
- utils::IdGenerator::getIdGenerator()->generate(), logging::LoggerFactory<SiteToSiteProvenanceReportingTask>::getLogger()) {
+ utils::IdGenerator::getIdGenerator()->generate(), logging::LoggerFactory<SiteToSiteProvenanceReportingTask>::getLogger()) {
this->setTriggerWhenEmpty(true);
batch_size_ = 100;
}
diff --git a/libminifi/src/core/FlowConfiguration.cpp b/libminifi/src/core/FlowConfiguration.cpp
index d2c21ee9c..379c18fe3 100644
--- a/libminifi/src/core/FlowConfiguration.cpp
+++ b/libminifi/src/core/FlowConfiguration.cpp
@@ -80,8 +80,10 @@ std::unique_ptr<core::Processor> FlowConfiguration::createProcessor(const std::s
return processor;
}
-std::unique_ptr<core::reporting::SiteToSiteProvenanceReportingTask> FlowConfiguration::createProvenanceReportTask() {
- return std::make_unique<org::apache::nifi::minifi::core::reporting::SiteToSiteProvenanceReportingTask>(this->configuration_);
+std::unique_ptr<core::Processor> FlowConfiguration::createProvenanceReportTask() {
+ auto processor = std::make_unique<core::Processor>("", std::make_unique<core::reporting::SiteToSiteProvenanceReportingTask>(this->configuration_));
+ processor->initialize();
+ return processor;
}
std::unique_ptr<core::ProcessGroup> FlowConfiguration::updateFromPayload(const std::string& url, const std::string& yamlConfigPayload, const std::optional<std::string>& flow_id) {
diff --git a/libminifi/src/core/flow/StructuredConfiguration.cpp b/libminifi/src/core/flow/StructuredConfiguration.cpp
index 55411616d..cb5a696bb 100644
--- a/libminifi/src/core/flow/StructuredConfiguration.cpp
+++ b/libminifi/src/core/flow/StructuredConfiguration.cpp
@@ -582,9 +582,7 @@ void StructuredConfiguration::parseProvenanceReporting(const Node& node, core::P
return;
}
- auto report_task_impl = createProvenanceReportTask();
- auto* reportTask = report_task_impl.get();
- auto report_task_wrapper = std::make_unique<core::Processor>("", std::move(report_task_impl));
+ auto report_task = createProvenanceReportTask();
checkRequiredField(node, schema_.scheduling_strategy);
auto schedulingStrategyStr = node[schema_.scheduling_strategy].getString().value();
@@ -593,11 +591,11 @@ void StructuredConfiguration::parseProvenanceReporting(const Node& node, core::P
if (auto scheduling_period = utils::timeutils::StringToDuration<std::chrono::nanoseconds>(schedulingPeriodStr)) {
logger_->log_debug("ProvenanceReportingTask schedulingPeriod {}", scheduling_period);
- report_task_wrapper->setSchedulingPeriod(*scheduling_period);
+ report_task->setSchedulingPeriod(*scheduling_period);
}
if (schedulingStrategyStr == "TIMER_DRIVEN") {
- report_task_wrapper->setSchedulingStrategy(core::TIMER_DRIVEN);
+ report_task->setSchedulingStrategy(core::TIMER_DRIVEN);
logger_->log_debug("ProvenanceReportingTask scheduling strategy {}", schedulingStrategyStr);
} else {
throw std::invalid_argument("Invalid scheduling strategy " + schedulingStrategyStr);
@@ -610,14 +608,14 @@ void StructuredConfiguration::parseProvenanceReporting(const Node& node, core::P
if (auto port = parsing::parseIntegral<int64_t>(portStr); port && !hostStr.empty()) {
logger_->log_debug("ProvenanceReportingTask port {}", *port);
std::string url = hostStr + ":" + portStr;
- reportTask->setURL(url);
+ report_task->getImpl<core::reporting::SiteToSiteProvenanceReportingTask>().setURL(url);
}
}
if (node["url"]) {
auto urlStr = node["url"].getString().value();
if (!urlStr.empty()) {
- reportTask->setURL(urlStr);
+ report_task->getImpl<core::reporting::SiteToSiteProvenanceReportingTask>().setURL(urlStr);
logger_->log_debug("ProvenanceReportingTask URL {}", urlStr);
}
}
@@ -628,17 +626,15 @@ void StructuredConfiguration::parseProvenanceReporting(const Node& node, core::P
logger_->log_debug("ProvenanceReportingTask port uuid {}", portUUIDStr);
port_uuid = portUUIDStr;
- reportTask->setPortUUID(port_uuid);
+ report_task->getImpl<core::reporting::SiteToSiteProvenanceReportingTask>().setPortUUID(port_uuid);
if (auto batch_size = parsing::parseIntegral<int>(batchSizeStr)) {
- reportTask->setBatchSize(*batch_size);
+ report_task->getImpl<core::reporting::SiteToSiteProvenanceReportingTask>().setBatchSize(*batch_size);
}
- report_task_wrapper->initialize();
-
// add processor to parent
- report_task_wrapper->setScheduledState(core::RUNNING);
- parent_group->addProcessor(std::move(report_task_wrapper));
+ report_task->setScheduledState(core::RUNNING);
+ parent_group->addProcessor(std::move(report_task));
}
void StructuredConfiguration::parseControllerServices(const Node& controller_services_node, core::ProcessGroup* parent_group) {
(nifi-minifi-cpp) 25/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit f5cd565c7f4eb49a3288edc3592021ae66e63bfe
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Jun 30 09:10:53 2025 +0200
Rebase fix
---
core-framework/include/core/ProcessContext.h | 2 +-
extensions/standard-processors/tests/unit/TailFileTests.cpp | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/core-framework/include/core/ProcessContext.h b/core-framework/include/core/ProcessContext.h
index e3b4c1d8d..83c80b4ed 100644
--- a/core-framework/include/core/ProcessContext.h
+++ b/core-framework/include/core/ProcessContext.h
@@ -62,7 +62,7 @@ class ProcessContextImpl : public core::VariableRegistryImpl, public virtual Pro
const ProcessorInfo& getProcessorInfo() const override { return *info_; }
- virtual bool hasNonEmptyProperty(std::string_view name) const override;
+ bool hasNonEmptyProperty(std::string_view name) const override;
bool isRunning() const override;
nonstd::expected<std::string, std::error_code> getProperty(std::string_view name, const FlowFile*) const override;
diff --git a/extensions/standard-processors/tests/unit/TailFileTests.cpp b/extensions/standard-processors/tests/unit/TailFileTests.cpp
index bae6ac500..916e24605 100644
--- a/extensions/standard-processors/tests/unit/TailFileTests.cpp
+++ b/extensions/standard-processors/tests/unit/TailFileTests.cpp
@@ -1807,7 +1807,7 @@ TEST_CASE("TailFile honors batch size for maximum lines processed", "[batchSize]
TEST_CASE("Result mode tests") {
LogTestController::getInstance().setTrace<minifi::processors::TailFile>();
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<minifi::processors::TailFile>("TailFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<minifi::processors::TailFile>("TailFile"));
auto tail_file = test_controller.getProcessor();
auto temp_file_path = test_controller.createTempDirectory() / TMP_FILE;
(nifi-minifi-cpp) 26/37: Windows fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit eed06552a6dd68b2c2c82219c08596bb79e9077c
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Wed Jul 2 15:29:13 2025 +0200
Windows fix
---
extensions/smb/tests/ListSmbTests.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/extensions/smb/tests/ListSmbTests.cpp b/extensions/smb/tests/ListSmbTests.cpp
index d7b197831..7269df7e3 100644
--- a/extensions/smb/tests/ListSmbTests.cpp
+++ b/extensions/smb/tests/ListSmbTests.cpp
@@ -53,7 +53,7 @@ bool checkForFlowFileWithAttributes(const std::vector<std::shared_ptr<core::Flow
}
TEST_CASE("ListSmb tests") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ListSmb>("ListSmb")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ListSmb>("ListSmb")};
const auto list_smb = controller.getProcessor();
auto smb_connection_node = controller.plan->addController("MockSmbConnectionControllerService", "smb_connection_controller_service");
(nifi-minifi-cpp) 33/37: Build fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit c0dfe3a9ecfb20104f1d5d3261bb67f3c7e3a023
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Jul 21 12:39:36 2025 +0200
Build fix
---
libminifi/src/core/flow/StructuredConnectionParser.cpp | 3 ++-
minifi-api/include/minifi-cpp/core/ProcessorDescriptor.h | 4 ++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/libminifi/src/core/flow/StructuredConnectionParser.cpp b/libminifi/src/core/flow/StructuredConnectionParser.cpp
index 4867e54df..870911580 100644
--- a/libminifi/src/core/flow/StructuredConnectionParser.cpp
+++ b/libminifi/src/core/flow/StructuredConnectionParser.cpp
@@ -41,7 +41,8 @@ void StructuredConnectionParser::addFunnelRelationshipToConnection(minifi::Conne
return;
}
- if (typeid(minifi::Funnel) == typeid(processor->getImpl())) {
+ auto& impl = processor->getImpl();
+ if (typeid(minifi::Funnel) == typeid(impl)) {
addNewRelationshipToConnection(minifi::Funnel::Success.name, connection);
}
}
diff --git a/minifi-api/include/minifi-cpp/core/ProcessorDescriptor.h b/minifi-api/include/minifi-cpp/core/ProcessorDescriptor.h
index fd942605c..e719b9252 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessorDescriptor.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessorDescriptor.h
@@ -16,9 +16,9 @@
*/
#pragma once
-#include "minifi-cpp/core/RelationshipDefinition.h"
-#include "minifi-cpp/core/PropertyDefinition.h"
#include <span>
+#include "minifi-cpp/core/PropertyDefinition.h"
+#include "minifi-cpp/core/RelationshipDefinition.h"
namespace org::apache::nifi::minifi::core {
(nifi-minifi-cpp) 28/37: Review changes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit bf75a8ede3d47180297c9438f31d43d2434655f5
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Jul 7 11:25:55 2025 +0200
Review changes
---
core-framework/include/core/ProcessorFactory.h | 4 ++--
core-framework/include/core/ProcessorImpl.h | 4 ++--
core-framework/src/core/ProcessorImpl.cpp | 10 +++++-----
extensions/aws/processors/DeleteS3Object.h | 4 ++--
extensions/aws/processors/FetchS3Object.h | 4 ++--
extensions/aws/processors/ListS3.h | 4 ++--
extensions/aws/processors/PutS3Object.h | 4 ++--
extensions/aws/processors/S3Processor.cpp | 4 ++--
extensions/aws/processors/S3Processor.h | 2 +-
.../azure/processors/AzureBlobStorageProcessorBase.h | 4 ++--
.../processors/AzureBlobStorageSingleBlobProcessorBase.h | 4 ++--
.../processors/AzureDataLakeStorageFileProcessorBase.h | 4 ++--
.../azure/processors/AzureDataLakeStorageProcessorBase.h | 4 ++--
extensions/azure/processors/DeleteAzureBlobStorage.h | 8 ++++----
extensions/azure/processors/DeleteAzureDataLakeStorage.h | 4 ++--
extensions/azure/processors/FetchAzureBlobStorage.h | 8 ++++----
extensions/azure/processors/FetchAzureDataLakeStorage.h | 4 ++--
extensions/azure/processors/ListAzureBlobStorage.h | 8 ++++----
extensions/azure/processors/ListAzureDataLakeStorage.h | 4 ++--
extensions/azure/processors/PutAzureBlobStorage.h | 8 ++++----
extensions/azure/processors/PutAzureDataLakeStorage.h | 4 ++--
extensions/civetweb/processors/ListenHTTP.h | 4 ++--
extensions/grafana-loki/PushGrafanaLoki.h | 6 +++---
extensions/libarchive/MergeContent.h | 4 ++--
extensions/llamacpp/processors/RunLlamaCppInference.h | 4 ++--
extensions/mqtt/processors/PublishMQTT.h | 4 ++--
extensions/pdh/PerformanceDataMonitor.h | 4 ++--
extensions/python/ExecutePythonProcessor.cpp | 2 +-
extensions/python/ExecutePythonProcessor.h | 6 +++---
extensions/python/PythonObjectFactory.h | 4 ++--
extensions/sftp/processors/PutSFTP.cpp | 4 ++--
extensions/sftp/processors/PutSFTP.h | 2 +-
extensions/sftp/processors/SFTPProcessorBase.cpp | 4 ++--
extensions/sftp/processors/SFTPProcessorBase.h | 2 +-
extensions/smb/tests/ListAndFetchSmbTests.cpp | 4 ++--
extensions/standard-processors/processors/GetFile.h | 4 ++--
extensions/standard-processors/processors/InvokeHTTP.h | 4 ++--
extensions/standard-processors/processors/LogAttribute.h | 4 ++--
extensions/standard-processors/processors/RouteText.cpp | 14 ++++----------
.../standard-processors/tests/unit/DefragmentTextTests.cpp | 4 ++--
extensions/systemd/ConsumeJournald.cpp | 2 +-
extensions/systemd/ConsumeJournald.h | 2 +-
extensions/windows-event-log/ConsumeWindowsEventLog.cpp | 4 ++--
extensions/windows-event-log/ConsumeWindowsEventLog.h | 2 +-
libminifi/test/libtest/unit/MockClasses.h | 4 ++--
minifi-api/include/minifi-cpp/core/ProcessContext.h | 5 -----
minifi-api/include/minifi-cpp/core/ProcessorFactory.h | 2 +-
47 files changed, 99 insertions(+), 110 deletions(-)
diff --git a/core-framework/include/core/ProcessorFactory.h b/core-framework/include/core/ProcessorFactory.h
index 17a42b78d..bbba22664 100644
--- a/core-framework/include/core/ProcessorFactory.h
+++ b/core-framework/include/core/ProcessorFactory.h
@@ -41,8 +41,8 @@ class ProcessorFactoryImpl : public ProcessorFactory {
return group_name_;
}
- std::unique_ptr<ProcessorApi> create(ProcessorMetadata info) override {
- return std::make_unique<T>(info);
+ std::unique_ptr<ProcessorApi> create(ProcessorMetadata metadata) override {
+ return std::make_unique<T>(metadata);
}
std::string getClassName() const override {
diff --git a/core-framework/include/core/ProcessorImpl.h b/core-framework/include/core/ProcessorImpl.h
index 6b14e5e3d..987bb7586 100644
--- a/core-framework/include/core/ProcessorImpl.h
+++ b/core-framework/include/core/ProcessorImpl.h
@@ -75,7 +75,7 @@ class ProcessSessionFactory;
class ProcessorImpl : public virtual ProcessorApi {
public:
- explicit ProcessorImpl(ProcessorMetadata info);
+ explicit ProcessorImpl(ProcessorMetadata metadata);
ProcessorImpl(const ProcessorImpl&) = delete;
ProcessorImpl(ProcessorImpl&&) = delete;
@@ -142,7 +142,7 @@ class ProcessorImpl : public virtual ProcessorApi {
void notifyStop() override {
}
- ProcessorMetadata info_;
+ ProcessorMetadata metadata_;
std::atomic<bool> trigger_when_empty_;
diff --git a/core-framework/src/core/ProcessorImpl.cpp b/core-framework/src/core/ProcessorImpl.cpp
index 68f5885bd..bda6c6910 100644
--- a/core-framework/src/core/ProcessorImpl.cpp
+++ b/core-framework/src/core/ProcessorImpl.cpp
@@ -44,11 +44,11 @@ using namespace std::literals::chrono_literals;
namespace org::apache::nifi::minifi::core {
-ProcessorImpl::ProcessorImpl(ProcessorMetadata info)
- : info_(std::move(info)),
+ProcessorImpl::ProcessorImpl(ProcessorMetadata metadata)
+ : metadata_(std::move(metadata)),
trigger_when_empty_(false),
metrics_(std::make_shared<ProcessorMetricsImpl>(*this)),
- logger_(info_.logger) {
+ logger_(metadata_.logger) {
logger_->log_debug("Processor {} created with uuid {}", getName(), getUUIDStr());
}
@@ -65,11 +65,11 @@ void ProcessorImpl::restore(const std::shared_ptr<FlowFile>& /*file*/) {
}
std::string ProcessorImpl::getName() const {
- return info_.name;
+ return metadata_.name;
}
utils::Identifier ProcessorImpl::getUUID() const {
- return info_.uuid;
+ return metadata_.uuid;
}
utils::SmallString<36> ProcessorImpl::getUUIDStr() const {
diff --git a/extensions/aws/processors/DeleteS3Object.h b/extensions/aws/processors/DeleteS3Object.h
index f723c58fd..1aa9c8075 100644
--- a/extensions/aws/processors/DeleteS3Object.h
+++ b/extensions/aws/processors/DeleteS3Object.h
@@ -74,8 +74,8 @@ class DeleteS3Object : public S3Processor { // NOLINT(cppcoreguidelines-special
private:
friend class ::FlowProcessorS3TestsFixture<DeleteS3Object>;
- explicit DeleteS3Object(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : S3Processor(info, std::move(s3_request_sender)) {
+ explicit DeleteS3Object(core::ProcessorMetadata metadata, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : S3Processor(metadata, std::move(s3_request_sender)) {
}
std::optional<aws::s3::DeleteObjectRequestParameters> buildDeleteS3RequestParams(
diff --git a/extensions/aws/processors/FetchS3Object.h b/extensions/aws/processors/FetchS3Object.h
index 19d9529bf..c703821fc 100644
--- a/extensions/aws/processors/FetchS3Object.h
+++ b/extensions/aws/processors/FetchS3Object.h
@@ -85,8 +85,8 @@ class FetchS3Object : public S3Processor { // NOLINT(cppcoreguidelines-special-
private:
friend class ::FlowProcessorS3TestsFixture<FetchS3Object>;
- explicit FetchS3Object(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : S3Processor(info, std::move(s3_request_sender)) {
+ explicit FetchS3Object(core::ProcessorMetadata metadata, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : S3Processor(metadata, std::move(s3_request_sender)) {
}
std::optional<aws::s3::GetObjectRequestParameters> buildFetchS3RequestParams(
diff --git a/extensions/aws/processors/ListS3.h b/extensions/aws/processors/ListS3.h
index 4c9d64fb1..f54810d40 100644
--- a/extensions/aws/processors/ListS3.h
+++ b/extensions/aws/processors/ListS3.h
@@ -95,8 +95,8 @@ class ListS3 : public S3Processor { // NOLINT(cppcoreguidelines-special-member-
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
using S3Processor::S3Processor;
- explicit ListS3(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : S3Processor(info, std::move(s3_request_sender)) {
+ explicit ListS3(core::ProcessorMetadata metadata, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : S3Processor(metadata, std::move(s3_request_sender)) {
}
~ListS3() override = default;
diff --git a/extensions/aws/processors/PutS3Object.h b/extensions/aws/processors/PutS3Object.h
index 03647b5da..5fe86bf0d 100644
--- a/extensions/aws/processors/PutS3Object.h
+++ b/extensions/aws/processors/PutS3Object.h
@@ -194,8 +194,8 @@ class PutS3Object : public S3Processor { // NOLINT(cppcoreguidelines-special-me
friend class ::FlowProcessorS3TestsFixture<PutS3Object>;
- explicit PutS3Object(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : S3Processor(info, std::move(s3_request_sender)) {
+ explicit PutS3Object(core::ProcessorMetadata metadata, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : S3Processor(metadata, std::move(s3_request_sender)) {
}
virtual uint64_t getMinPartSize() const {
diff --git a/extensions/aws/processors/S3Processor.cpp b/extensions/aws/processors/S3Processor.cpp
index 532bb6012..44e08e08e 100644
--- a/extensions/aws/processors/S3Processor.cpp
+++ b/extensions/aws/processors/S3Processor.cpp
@@ -32,8 +32,8 @@
namespace org::apache::nifi::minifi::aws::processors {
-S3Processor::S3Processor(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : AwsProcessor(std::move(info)),
+S3Processor::S3Processor(core::ProcessorMetadata metadata, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : AwsProcessor(std::move(metadata)),
s3_wrapper_(std::move(s3_request_sender)) {
}
diff --git a/extensions/aws/processors/S3Processor.h b/extensions/aws/processors/S3Processor.h
index a5f966b5d..6d31d1eee 100644
--- a/extensions/aws/processors/S3Processor.h
+++ b/extensions/aws/processors/S3Processor.h
@@ -56,7 +56,7 @@ class S3Processor : public AwsProcessor {
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
protected:
- explicit S3Processor(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender);
+ explicit S3Processor(core::ProcessorMetadata metadata, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender);
aws::s3::S3Wrapper s3_wrapper_;
};
diff --git a/extensions/azure/processors/AzureBlobStorageProcessorBase.h b/extensions/azure/processors/AzureBlobStorageProcessorBase.h
index b40db3aad..32eb7f793 100644
--- a/extensions/azure/processors/AzureBlobStorageProcessorBase.h
+++ b/extensions/azure/processors/AzureBlobStorageProcessorBase.h
@@ -89,9 +89,9 @@ class AzureBlobStorageProcessorBase : public AzureStorageProcessorBase {
protected:
explicit AzureBlobStorageProcessorBase(
- core::ProcessorMetadata info,
+ core::ProcessorMetadata metadata,
std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureStorageProcessorBase(info),
+ : AzureStorageProcessorBase(metadata),
azure_blob_storage_(std::move(blob_storage_client)) {
}
diff --git a/extensions/azure/processors/AzureBlobStorageSingleBlobProcessorBase.h b/extensions/azure/processors/AzureBlobStorageSingleBlobProcessorBase.h
index bb1466249..0e2b08466 100644
--- a/extensions/azure/processors/AzureBlobStorageSingleBlobProcessorBase.h
+++ b/extensions/azure/processors/AzureBlobStorageSingleBlobProcessorBase.h
@@ -41,9 +41,9 @@ class AzureBlobStorageSingleBlobProcessorBase : public AzureBlobStorageProcessor
protected:
explicit AzureBlobStorageSingleBlobProcessorBase(
- core::ProcessorMetadata info,
+ core::ProcessorMetadata metadata,
std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageProcessorBase(info, std::move(blob_storage_client)) {
+ : AzureBlobStorageProcessorBase(metadata, std::move(blob_storage_client)) {
}
bool setBlobOperationParameters(
diff --git a/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h b/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
index 55d522ff5..f45608319 100644
--- a/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
+++ b/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
@@ -41,8 +41,8 @@ class AzureDataLakeStorageFileProcessorBase : public AzureDataLakeStorageProcess
~AzureDataLakeStorageFileProcessorBase() override = default;
protected:
- explicit AzureDataLakeStorageFileProcessorBase(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageProcessorBase(info, std::move(data_lake_storage_client)) {
+ explicit AzureDataLakeStorageFileProcessorBase(core::ProcessorMetadata metadata, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageProcessorBase(metadata, std::move(data_lake_storage_client)) {
}
bool setFileOperationCommonParameters(
diff --git a/extensions/azure/processors/AzureDataLakeStorageProcessorBase.h b/extensions/azure/processors/AzureDataLakeStorageProcessorBase.h
index 35b4383dd..3a3e2b07b 100644
--- a/extensions/azure/processors/AzureDataLakeStorageProcessorBase.h
+++ b/extensions/azure/processors/AzureDataLakeStorageProcessorBase.h
@@ -59,8 +59,8 @@ class AzureDataLakeStorageProcessorBase : public AzureStorageProcessorBase {
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& sessionFactory) override;
protected:
- explicit AzureDataLakeStorageProcessorBase(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureStorageProcessorBase(info),
+ explicit AzureDataLakeStorageProcessorBase(core::ProcessorMetadata metadata, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureStorageProcessorBase(metadata),
azure_data_lake_storage_(std::move(data_lake_storage_client)) {
}
diff --git a/extensions/azure/processors/DeleteAzureBlobStorage.h b/extensions/azure/processors/DeleteAzureBlobStorage.h
index bb5709179..890d9f19d 100644
--- a/extensions/azure/processors/DeleteAzureBlobStorage.h
+++ b/extensions/azure/processors/DeleteAzureBlobStorage.h
@@ -62,8 +62,8 @@ class DeleteAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorB
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit DeleteAzureBlobStorage(core::ProcessorMetadata info)
- : DeleteAzureBlobStorage(info, nullptr) {
+ explicit DeleteAzureBlobStorage(core::ProcessorMetadata metadata)
+ : DeleteAzureBlobStorage(metadata, nullptr) {
}
void initialize() override;
@@ -73,8 +73,8 @@ class DeleteAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorB
private:
friend class ::AzureBlobStorageTestsFixture<DeleteAzureBlobStorage>;
- explicit DeleteAzureBlobStorage(core::ProcessorMetadata info, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageSingleBlobProcessorBase(info, std::move(blob_storage_client)) {
+ explicit DeleteAzureBlobStorage(core::ProcessorMetadata metadata, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
+ : AzureBlobStorageSingleBlobProcessorBase(metadata, std::move(blob_storage_client)) {
}
std::optional<storage::DeleteAzureBlobStorageParameters> buildDeleteAzureBlobStorageParameters(
diff --git a/extensions/azure/processors/DeleteAzureDataLakeStorage.h b/extensions/azure/processors/DeleteAzureDataLakeStorage.h
index 88024909a..6f231c05f 100644
--- a/extensions/azure/processors/DeleteAzureDataLakeStorage.h
+++ b/extensions/azure/processors/DeleteAzureDataLakeStorage.h
@@ -58,8 +58,8 @@ class DeleteAzureDataLakeStorage final : public AzureDataLakeStorageFileProcesso
private:
friend class ::AzureDataLakeStorageTestsFixture<DeleteAzureDataLakeStorage>;
- explicit DeleteAzureDataLakeStorage(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageFileProcessorBase(info, std::move(data_lake_storage_client)) {
+ explicit DeleteAzureDataLakeStorage(core::ProcessorMetadata metadata, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageFileProcessorBase(metadata, std::move(data_lake_storage_client)) {
}
std::optional<storage::DeleteAzureDataLakeStorageParameters> buildDeleteParameters(core::ProcessContext& context, const core::FlowFile& flow_file);
diff --git a/extensions/azure/processors/FetchAzureBlobStorage.h b/extensions/azure/processors/FetchAzureBlobStorage.h
index d64ec37dd..3270becc4 100644
--- a/extensions/azure/processors/FetchAzureBlobStorage.h
+++ b/extensions/azure/processors/FetchAzureBlobStorage.h
@@ -68,8 +68,8 @@ class FetchAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorBa
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit FetchAzureBlobStorage(core::ProcessorMetadata info)
- : FetchAzureBlobStorage(info, nullptr) {
+ explicit FetchAzureBlobStorage(core::ProcessorMetadata metadata)
+ : FetchAzureBlobStorage(metadata, nullptr) {
}
void initialize() override;
@@ -78,8 +78,8 @@ class FetchAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorBa
private:
friend class ::AzureBlobStorageTestsFixture<FetchAzureBlobStorage>;
- explicit FetchAzureBlobStorage(core::ProcessorMetadata info, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageSingleBlobProcessorBase(info, std::move(blob_storage_client)) {
+ explicit FetchAzureBlobStorage(core::ProcessorMetadata metadata, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
+ : AzureBlobStorageSingleBlobProcessorBase(metadata, std::move(blob_storage_client)) {
}
std::optional<storage::FetchAzureBlobStorageParameters> buildFetchAzureBlobStorageParameters(
diff --git a/extensions/azure/processors/FetchAzureDataLakeStorage.h b/extensions/azure/processors/FetchAzureDataLakeStorage.h
index 5626e8c28..d2a8e8565 100644
--- a/extensions/azure/processors/FetchAzureDataLakeStorage.h
+++ b/extensions/azure/processors/FetchAzureDataLakeStorage.h
@@ -79,8 +79,8 @@ class FetchAzureDataLakeStorage final : public AzureDataLakeStorageFileProcessor
private:
friend class ::AzureDataLakeStorageTestsFixture<FetchAzureDataLakeStorage>;
- explicit FetchAzureDataLakeStorage(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageFileProcessorBase(info, std::move(data_lake_storage_client)) {
+ explicit FetchAzureDataLakeStorage(core::ProcessorMetadata metadata, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageFileProcessorBase(metadata, std::move(data_lake_storage_client)) {
}
std::optional<storage::FetchAzureDataLakeStorageParameters> buildFetchParameters(core::ProcessContext& context, const core::FlowFile& flow_file);
diff --git a/extensions/azure/processors/ListAzureBlobStorage.h b/extensions/azure/processors/ListAzureBlobStorage.h
index 96ce71c7c..cfe630bf6 100644
--- a/extensions/azure/processors/ListAzureBlobStorage.h
+++ b/extensions/azure/processors/ListAzureBlobStorage.h
@@ -64,12 +64,12 @@ class ListAzureBlobStorage final : public AzureBlobStorageProcessorBase {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit ListAzureBlobStorage(core::ProcessorMetadata info)
- : ListAzureBlobStorage(info, nullptr) {
+ explicit ListAzureBlobStorage(core::ProcessorMetadata metadata)
+ : ListAzureBlobStorage(metadata, nullptr) {
}
- explicit ListAzureBlobStorage(core::ProcessorMetadata info, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageProcessorBase(info, std::move(blob_storage_client)) {
+ explicit ListAzureBlobStorage(core::ProcessorMetadata metadata, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
+ : AzureBlobStorageProcessorBase(metadata, std::move(blob_storage_client)) {
}
void initialize() override;
diff --git a/extensions/azure/processors/ListAzureDataLakeStorage.h b/extensions/azure/processors/ListAzureDataLakeStorage.h
index fa45cf168..c444c98c5 100644
--- a/extensions/azure/processors/ListAzureDataLakeStorage.h
+++ b/extensions/azure/processors/ListAzureDataLakeStorage.h
@@ -84,8 +84,8 @@ class ListAzureDataLakeStorage final : public AzureDataLakeStorageProcessorBase
private:
friend class ::ListAzureDataLakeStorageTestsFixture;
- explicit ListAzureDataLakeStorage(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageProcessorBase(info, std::move(data_lake_storage_client)) {
+ explicit ListAzureDataLakeStorage(core::ProcessorMetadata metadata, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageProcessorBase(metadata, std::move(data_lake_storage_client)) {
}
std::optional<storage::ListAzureDataLakeStorageParameters> buildListParameters(core::ProcessContext &context);
diff --git a/extensions/azure/processors/PutAzureBlobStorage.h b/extensions/azure/processors/PutAzureBlobStorage.h
index 695f4853a..4c1f9d6ab 100644
--- a/extensions/azure/processors/PutAzureBlobStorage.h
+++ b/extensions/azure/processors/PutAzureBlobStorage.h
@@ -65,8 +65,8 @@ class PutAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorBase
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PutAzureBlobStorage(core::ProcessorMetadata info)
- : PutAzureBlobStorage(info, nullptr) {
+ explicit PutAzureBlobStorage(core::ProcessorMetadata metadata)
+ : PutAzureBlobStorage(metadata, nullptr) {
}
void initialize() override;
@@ -107,8 +107,8 @@ class PutAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorBase
private:
friend class ::AzureBlobStorageTestsFixture<PutAzureBlobStorage>;
- explicit PutAzureBlobStorage(core::ProcessorMetadata info, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageSingleBlobProcessorBase(info, std::move(blob_storage_client)) {
+ explicit PutAzureBlobStorage(core::ProcessorMetadata metadata, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
+ : AzureBlobStorageSingleBlobProcessorBase(metadata, std::move(blob_storage_client)) {
}
std::optional<storage::PutAzureBlobStorageParameters> buildPutAzureBlobStorageParameters(core::ProcessContext &context, const core::FlowFile& flow_file);
diff --git a/extensions/azure/processors/PutAzureDataLakeStorage.h b/extensions/azure/processors/PutAzureDataLakeStorage.h
index f1684c003..e1bac70cc 100644
--- a/extensions/azure/processors/PutAzureDataLakeStorage.h
+++ b/extensions/azure/processors/PutAzureDataLakeStorage.h
@@ -95,8 +95,8 @@ class PutAzureDataLakeStorage final : public AzureDataLakeStorageFileProcessorBa
std::shared_ptr<core::logging::Logger> logger_;
};
- explicit PutAzureDataLakeStorage(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageFileProcessorBase(info, std::move(data_lake_storage_client)) {
+ explicit PutAzureDataLakeStorage(core::ProcessorMetadata metadata, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageFileProcessorBase(metadata, std::move(data_lake_storage_client)) {
}
std::optional<storage::PutAzureDataLakeStorageParameters> buildUploadParameters(core::ProcessContext &context, const core::FlowFile& flow_file);
diff --git a/extensions/civetweb/processors/ListenHTTP.h b/extensions/civetweb/processors/ListenHTTP.h
index 199b2155b..06da14e1b 100644
--- a/extensions/civetweb/processors/ListenHTTP.h
+++ b/extensions/civetweb/processors/ListenHTTP.h
@@ -55,8 +55,8 @@ class ListenHTTP : public core::ProcessorImpl {
public:
friend struct ::org::apache::nifi::minifi::test::ListenHTTPTestAccessor;
- explicit ListenHTTP(core::ProcessorMetadata info)
- : ProcessorImpl(info) {
+ explicit ListenHTTP(core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata) {
callbacks_.log_message = &logMessage;
callbacks_.log_access = &logAccess;
}
diff --git a/extensions/grafana-loki/PushGrafanaLoki.h b/extensions/grafana-loki/PushGrafanaLoki.h
index 5f8f914cb..bd64aa021 100644
--- a/extensions/grafana-loki/PushGrafanaLoki.h
+++ b/extensions/grafana-loki/PushGrafanaLoki.h
@@ -34,9 +34,9 @@ namespace org::apache::nifi::minifi::extensions::grafana::loki {
class PushGrafanaLoki : public core::ProcessorImpl {
public:
- explicit PushGrafanaLoki(core::ProcessorMetadata info)
- : ProcessorImpl(info),
- log_batch_(info.logger) {
+ explicit PushGrafanaLoki(core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata),
+ log_batch_(metadata.logger) {
}
~PushGrafanaLoki() override = default;
diff --git a/extensions/libarchive/MergeContent.h b/extensions/libarchive/MergeContent.h
index 0dbed6f40..2853f0c5d 100644
--- a/extensions/libarchive/MergeContent.h
+++ b/extensions/libarchive/MergeContent.h
@@ -294,8 +294,8 @@ class KeepAllUniqueAttributesMerger: public AttributeMerger {
*/
class MergeContent : public processors::BinFiles {
public:
- explicit MergeContent(core::ProcessorMetadata info)
- : processors::BinFiles(info) {
+ explicit MergeContent(core::ProcessorMetadata metadata)
+ : processors::BinFiles(metadata) {
mergeStrategy_ = merge_content_options::MERGE_STRATEGY_DEFRAGMENT;
mergeFormat_ = merge_content_options::MERGE_FORMAT_CONCAT_VALUE;
delimiterStrategy_ = merge_content_options::DELIMITER_STRATEGY_FILENAME;
diff --git a/extensions/llamacpp/processors/RunLlamaCppInference.h b/extensions/llamacpp/processors/RunLlamaCppInference.h
index 13eafa4e6..c2d11fe45 100644
--- a/extensions/llamacpp/processors/RunLlamaCppInference.h
+++ b/extensions/llamacpp/processors/RunLlamaCppInference.h
@@ -63,8 +63,8 @@ class RunLlamaCppInferenceMetrics : public core::ProcessorMetricsImpl {
class RunLlamaCppInference : public core::ProcessorImpl {
public:
- explicit RunLlamaCppInference(core::ProcessorMetadata info, LlamaContextProvider llama_context_provider = {})
- : core::ProcessorImpl(info),
+ explicit RunLlamaCppInference(core::ProcessorMetadata metadata, LlamaContextProvider llama_context_provider = {})
+ : core::ProcessorImpl(metadata),
llama_context_provider_(std::move(llama_context_provider)) {
metrics_ = gsl::make_not_null(std::make_shared<RunLlamaCppInferenceMetrics>(*this));
}
diff --git a/extensions/mqtt/processors/PublishMQTT.h b/extensions/mqtt/processors/PublishMQTT.h
index 3f252e6b9..a84fea851 100644
--- a/extensions/mqtt/processors/PublishMQTT.h
+++ b/extensions/mqtt/processors/PublishMQTT.h
@@ -39,8 +39,8 @@ namespace org::apache::nifi::minifi::processors {
class PublishMQTT : public processors::AbstractMQTTProcessor {
public:
- explicit PublishMQTT(core::ProcessorMetadata info)
- : processors::AbstractMQTTProcessor(info) {
+ explicit PublishMQTT(core::ProcessorMetadata metadata)
+ : processors::AbstractMQTTProcessor(metadata) {
metrics_ = gsl::make_not_null(std::make_shared<PublishMQTTMetrics>(*this, in_flight_message_counter_));
}
diff --git a/extensions/pdh/PerformanceDataMonitor.h b/extensions/pdh/PerformanceDataMonitor.h
index 08d6ddbe9..196ca3db9 100644
--- a/extensions/pdh/PerformanceDataMonitor.h
+++ b/extensions/pdh/PerformanceDataMonitor.h
@@ -49,8 +49,8 @@ class PerformanceDataMonitor final : public core::ProcessorImpl {
static constexpr const char* PRETTY_FORMAT_STR = "Pretty";
static constexpr const char* COMPACT_FORMAT_STR = "Compact";
- explicit PerformanceDataMonitor(core::ProcessorMetadata info)
- : ProcessorImpl(info), output_format_(OutputFormat::JSON), pretty_output_(false),
+ explicit PerformanceDataMonitor(core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata), output_format_(OutputFormat::JSON), pretty_output_(false),
decimal_places_(std::nullopt),
pdh_query_(nullptr), resource_consumption_counters_() {}
diff --git a/extensions/python/ExecutePythonProcessor.cpp b/extensions/python/ExecutePythonProcessor.cpp
index 48a58ad6f..b9ed72bc1 100644
--- a/extensions/python/ExecutePythonProcessor.cpp
+++ b/extensions/python/ExecutePythonProcessor.cpp
@@ -60,7 +60,7 @@ void ExecutePythonProcessor::initialize() {
ranges::transform(python_properties_, std::back_inserter(all_properties), &core::Property::getReference);
setSupportedProperties(all_properties);
setSupportedRelationships(Relationships);
- logger_->log_debug("Processor has already been initialized, returning...");
+ logger_->log_debug("Processor has been initialized.");
}
void ExecutePythonProcessor::initalizeThroughScriptEngine() {
diff --git a/extensions/python/ExecutePythonProcessor.h b/extensions/python/ExecutePythonProcessor.h
index 1e44b8cc3..e78623f68 100644
--- a/extensions/python/ExecutePythonProcessor.h
+++ b/extensions/python/ExecutePythonProcessor.h
@@ -40,12 +40,12 @@ namespace org::apache::nifi::minifi::extensions::python::processors {
class ExecutePythonProcessor : public core::ProcessorImpl {
public:
- explicit ExecutePythonProcessor(core::ProcessorMetadata info)
- : ProcessorImpl(info),
+ explicit ExecutePythonProcessor(core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata),
processor_initialized_(false),
python_dynamic_(false),
reload_on_script_change_(true) {
- python_logger_ = core::logging::LoggerFactory<ExecutePythonProcessor>::getAliasedLogger(getName(), info.uuid);
+ python_logger_ = core::logging::LoggerFactory<ExecutePythonProcessor>::getAliasedLogger(getName(), metadata.uuid);
}
EXTENSIONAPI static constexpr const char* Description = "Executes a script given the flow file and a process session. "
diff --git a/extensions/python/PythonObjectFactory.h b/extensions/python/PythonObjectFactory.h
index 4c75deb38..577f53f9a 100644
--- a/extensions/python/PythonObjectFactory.h
+++ b/extensions/python/PythonObjectFactory.h
@@ -45,8 +45,8 @@ class PythonObjectFactory : public org::apache::nifi::minifi::core::ProcessorFac
qualified_module_name_(std::move(qualified_module_name)) {
}
- std::unique_ptr<org::apache::nifi::minifi::core::ProcessorApi> create(org::apache::nifi::minifi::core::ProcessorMetadata info) override {
- auto obj = ProcessorFactoryImpl::create(info);
+ std::unique_ptr<org::apache::nifi::minifi::core::ProcessorApi> create(org::apache::nifi::minifi::core::ProcessorMetadata metadata) override {
+ auto obj = ProcessorFactoryImpl::create(metadata);
auto ptr = org::apache::nifi::minifi::utils::dynamic_unique_cast<org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor>(std::move(obj));
if (ptr == nullptr) {
return nullptr;
diff --git a/extensions/sftp/processors/PutSFTP.cpp b/extensions/sftp/processors/PutSFTP.cpp
index 5d0e836c0..26a6d4bf7 100644
--- a/extensions/sftp/processors/PutSFTP.cpp
+++ b/extensions/sftp/processors/PutSFTP.cpp
@@ -42,8 +42,8 @@ void PutSFTP::initialize() {
setSupportedRelationships(Relationships);
}
-PutSFTP::PutSFTP(core::ProcessorMetadata info)
- : SFTPProcessorBase(info),
+PutSFTP::PutSFTP(core::ProcessorMetadata metadata)
+ : SFTPProcessorBase(metadata),
create_directory_(false),
batch_size_(0),
reject_zero_byte_(false),
diff --git a/extensions/sftp/processors/PutSFTP.h b/extensions/sftp/processors/PutSFTP.h
index 2343bc4d7..92bc62411 100644
--- a/extensions/sftp/processors/PutSFTP.h
+++ b/extensions/sftp/processors/PutSFTP.h
@@ -51,7 +51,7 @@ class PutSFTP : public SFTPProcessorBase {
static constexpr std::string_view CONFLICT_RESOLUTION_FAIL = "FAIL";
static constexpr std::string_view CONFLICT_RESOLUTION_NONE = "NONE";
- explicit PutSFTP(core::ProcessorMetadata info);
+ explicit PutSFTP(core::ProcessorMetadata metadata);
~PutSFTP() override;
EXTENSIONAPI static constexpr const char* Description = "Sends FlowFiles to an SFTP Server";
diff --git a/extensions/sftp/processors/SFTPProcessorBase.cpp b/extensions/sftp/processors/SFTPProcessorBase.cpp
index c9d620233..a54fad7d2 100644
--- a/extensions/sftp/processors/SFTPProcessorBase.cpp
+++ b/extensions/sftp/processors/SFTPProcessorBase.cpp
@@ -43,8 +43,8 @@
namespace org::apache::nifi::minifi::processors {
-SFTPProcessorBase::SFTPProcessorBase(core::ProcessorMetadata info)
- : ProcessorImpl(info),
+SFTPProcessorBase::SFTPProcessorBase(core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata),
connection_timeout_(0),
data_timeout_(0),
strict_host_checking_(false),
diff --git a/extensions/sftp/processors/SFTPProcessorBase.h b/extensions/sftp/processors/SFTPProcessorBase.h
index a63c57d71..d68ce7aa9 100644
--- a/extensions/sftp/processors/SFTPProcessorBase.h
+++ b/extensions/sftp/processors/SFTPProcessorBase.h
@@ -40,7 +40,7 @@ namespace org::apache::nifi::minifi::processors {
class SFTPProcessorBase : public core::ProcessorImpl {
public:
- explicit SFTPProcessorBase(core::ProcessorMetadata info);
+ explicit SFTPProcessorBase(core::ProcessorMetadata metadata);
~SFTPProcessorBase() override;
static constexpr std::string_view PROXY_TYPE_DIRECT = "DIRECT";
diff --git a/extensions/smb/tests/ListAndFetchSmbTests.cpp b/extensions/smb/tests/ListAndFetchSmbTests.cpp
index d413ece1b..e75462ebe 100644
--- a/extensions/smb/tests/ListAndFetchSmbTests.cpp
+++ b/extensions/smb/tests/ListAndFetchSmbTests.cpp
@@ -65,8 +65,8 @@ TEST_CASE("ListSmb and FetchSmb work together") {
}
controller.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 1);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 1);
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
}
} // namespace org::apache::nifi::minifi::extensions::smb::test
diff --git a/extensions/standard-processors/processors/GetFile.h b/extensions/standard-processors/processors/GetFile.h
index 09fc01133..d255ba39f 100644
--- a/extensions/standard-processors/processors/GetFile.h
+++ b/extensions/standard-processors/processors/GetFile.h
@@ -84,8 +84,8 @@ class GetFileMetrics : public core::ProcessorMetricsImpl {
class GetFile : public core::ProcessorImpl {
public:
- explicit GetFile(core::ProcessorMetadata info)
- : ProcessorImpl(info) {
+ explicit GetFile(core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata) {
metrics_ = gsl::make_not_null(std::make_shared<GetFileMetrics>(*this));
}
~GetFile() override = default;
diff --git a/extensions/standard-processors/processors/InvokeHTTP.h b/extensions/standard-processors/processors/InvokeHTTP.h
index da4652a89..a8950057f 100644
--- a/extensions/standard-processors/processors/InvokeHTTP.h
+++ b/extensions/standard-processors/processors/InvokeHTTP.h
@@ -127,8 +127,8 @@ class InvokeHTTP : public core::ProcessorImpl {
EXTENSIONAPI static constexpr std::string_view REQUEST_URL = "invokehttp.request.url";
EXTENSIONAPI static constexpr std::string_view TRANSACTION_ID = "invokehttp.tx.id";
- explicit InvokeHTTP(core::ProcessorMetadata info)
- : ProcessorImpl(info) {
+ explicit InvokeHTTP(core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata) {
setTriggerWhenEmpty(true);
}
diff --git a/extensions/standard-processors/processors/LogAttribute.h b/extensions/standard-processors/processors/LogAttribute.h
index c47f414ff..55bc6c938 100644
--- a/extensions/standard-processors/processors/LogAttribute.h
+++ b/extensions/standard-processors/processors/LogAttribute.h
@@ -39,8 +39,8 @@ namespace org::apache::nifi::minifi::processors {
class LogAttribute : public core::ProcessorImpl {
public:
- explicit LogAttribute(core::ProcessorMetadata info)
- : ProcessorImpl(info) {
+ explicit LogAttribute(core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata) {
logger_->set_max_log_size(-1);
}
diff --git a/extensions/standard-processors/processors/RouteText.cpp b/extensions/standard-processors/processors/RouteText.cpp
index a8b5c5aad..5d005f6fb 100644
--- a/extensions/standard-processors/processors/RouteText.cpp
+++ b/extensions/standard-processors/processors/RouteText.cpp
@@ -54,16 +54,10 @@ void RouteText::onSchedule(core::ProcessContext& context, core::ProcessSessionFa
group_fallback_ = context.getProperty(GroupingFallbackValue).value_or("");
- {
- const auto static_relationships = RouteText::Relationships;
- std::vector<core::RelationshipDefinition> relationships(static_relationships.begin(), static_relationships.end());
-
- for (const auto& property_name : context.getDynamicPropertyKeys()) {
- core::RelationshipDefinition rel{property_name, "Dynamic Route"};
- dynamic_relationships_[property_name] = rel;
- relationships.push_back(rel);
- logger_->log_info("RouteText registered dynamic route '{}'", property_name);
- }
+ for (const auto& property_name : context.getDynamicPropertyKeys()) {
+ core::RelationshipDefinition rel{property_name, "Dynamic Route"};
+ dynamic_relationships_[property_name] = rel;
+ logger_->log_info("RouteText registered dynamic route '{}'", property_name);
}
}
diff --git a/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp b/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
index 1fb68b0c4..28f1579f0 100644
--- a/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
+++ b/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
@@ -323,8 +323,8 @@ class FragmentGenerator : public core::ProcessorImpl {
static constexpr bool IsSingleThreaded = false;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit FragmentGenerator(minifi::core::ProcessorMetadata info)
- : ProcessorImpl(info) {
+ explicit FragmentGenerator(minifi::core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata) {
}
void onTrigger(core::ProcessContext&, core::ProcessSession& session) override {
diff --git a/extensions/systemd/ConsumeJournald.cpp b/extensions/systemd/ConsumeJournald.cpp
index ca0cfa276..0a7043ed9 100644
--- a/extensions/systemd/ConsumeJournald.cpp
+++ b/extensions/systemd/ConsumeJournald.cpp
@@ -33,7 +33,7 @@ namespace org::apache::nifi::minifi::extensions::systemd {
namespace chr = std::chrono;
-ConsumeJournald::ConsumeJournald(core::ProcessorMetadata info, std::unique_ptr<libwrapper::LibWrapper>&& libwrapper)
+ConsumeJournald::ConsumeJournald(core::ProcessorMetadata metadata, std::unique_ptr<libwrapper::LibWrapper>&& libwrapper)
: core::ProcessorImpl{info}, libwrapper_{std::move(libwrapper)} {}
void ConsumeJournald::initialize() {
diff --git a/extensions/systemd/ConsumeJournald.h b/extensions/systemd/ConsumeJournald.h
index 07cd49dc4..32de33253 100644
--- a/extensions/systemd/ConsumeJournald.h
+++ b/extensions/systemd/ConsumeJournald.h
@@ -113,7 +113,7 @@ class ConsumeJournald final : public core::ProcessorImpl {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit ConsumeJournald(core::ProcessorMetadata info, std::unique_ptr<libwrapper::LibWrapper>&& = libwrapper::createLibWrapper());
+ explicit ConsumeJournald(core::ProcessorMetadata metadata, std::unique_ptr<libwrapper::LibWrapper>&& = libwrapper::createLibWrapper());
ConsumeJournald(const ConsumeJournald&) = delete;
ConsumeJournald(ConsumeJournald&&) = delete;
ConsumeJournald& operator=(const ConsumeJournald&) = delete;
diff --git a/extensions/windows-event-log/ConsumeWindowsEventLog.cpp b/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
index 391080840..d2bbbf6ea 100644
--- a/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
+++ b/extensions/windows-event-log/ConsumeWindowsEventLog.cpp
@@ -61,8 +61,8 @@ namespace org::apache::nifi::minifi::processors {
const int EVT_NEXT_TIMEOUT_MS = 500;
-ConsumeWindowsEventLog::ConsumeWindowsEventLog(core::ProcessorMetadata info)
- : core::ProcessorImpl(info) {
+ConsumeWindowsEventLog::ConsumeWindowsEventLog(core::ProcessorMetadata metadata)
+ : core::ProcessorImpl(metadata) {
char buff[MAX_COMPUTERNAME_LENGTH + 1];
DWORD size = sizeof(buff);
if (GetComputerName(buff, &size)) {
diff --git a/extensions/windows-event-log/ConsumeWindowsEventLog.h b/extensions/windows-event-log/ConsumeWindowsEventLog.h
index d47ab7ddf..263e9902c 100644
--- a/extensions/windows-event-log/ConsumeWindowsEventLog.h
+++ b/extensions/windows-event-log/ConsumeWindowsEventLog.h
@@ -80,7 +80,7 @@ class Bookmark;
class ConsumeWindowsEventLog : public core::ProcessorImpl {
public:
- explicit ConsumeWindowsEventLog(core::ProcessorMetadata info);
+ explicit ConsumeWindowsEventLog(core::ProcessorMetadata metadata);
~ConsumeWindowsEventLog() override;
diff --git a/libminifi/test/libtest/unit/MockClasses.h b/libminifi/test/libtest/unit/MockClasses.h
index bb55e7a30..0fde351b7 100644
--- a/libminifi/test/libtest/unit/MockClasses.h
+++ b/libminifi/test/libtest/unit/MockClasses.h
@@ -83,8 +83,8 @@ class MockControllerService : public minifi::core::controller::ControllerService
class MockProcessor : public minifi::core::ProcessorImpl {
public:
- explicit MockProcessor(minifi::core::ProcessorMetadata info)
- : ProcessorImpl(info) {
+ explicit MockProcessor(minifi::core::ProcessorMetadata metadata)
+ : ProcessorImpl(metadata) {
setTriggerWhenEmpty(true);
}
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContext.h b/minifi-api/include/minifi-cpp/core/ProcessContext.h
index fdc04a874..1e1e3e2b0 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContext.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContext.h
@@ -32,11 +32,6 @@
namespace org::apache::nifi::minifi::core {
-namespace detail {
-template<typename T>
-concept NotAFlowFile = !std::convertible_to<T &, const FlowFile &> && !std::convertible_to<T &, const std::shared_ptr<FlowFile> &>;
-} // namespace detail
-
class ProcessorInfo {
public:
virtual std::string getName() const = 0;
diff --git a/minifi-api/include/minifi-cpp/core/ProcessorFactory.h b/minifi-api/include/minifi-cpp/core/ProcessorFactory.h
index ed27ebf2e..258b857ed 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessorFactory.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessorFactory.h
@@ -28,7 +28,7 @@ class ProcessorApi;
class ProcessorFactory {
public:
- virtual std::unique_ptr<ProcessorApi> create(ProcessorMetadata info) = 0;
+ virtual std::unique_ptr<ProcessorApi> create(ProcessorMetadata metadata) = 0;
virtual std::string getGroupName() const = 0;
virtual std::string getClassName() const = 0;
(nifi-minifi-cpp) 07/37: Rebase fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit f1ce1feb1acd2089c4d27445f3c1de2b20879567
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Tue Apr 22 14:25:32 2025 +0200
Rebase fix
---
extensions/opc/include/opcbase.h | 2 --
extensions/opc/src/opcbase.cpp | 1 -
extensions/opc/src/putopc.cpp | 4 +--
extensions/opc/tests/FetchOpcProcessorTests.cpp | 18 +++++------
extensions/opc/tests/PutOpcProcessorTests.cpp | 36 +++++++++++-----------
extensions/python/PythonCreator.h | 10 +++---
.../tests/integration/InvokeHTTPTests.cpp | 4 +--
libminifi/src/core/ProcessContext.cpp | 27 ++++++++--------
8 files changed, 48 insertions(+), 54 deletions(-)
diff --git a/extensions/opc/include/opcbase.h b/extensions/opc/include/opcbase.h
index bdb79f558..982399125 100644
--- a/extensions/opc/include/opcbase.h
+++ b/extensions/opc/include/opcbase.h
@@ -82,8 +82,6 @@ class BaseOPCProcessor : public core::ProcessorImpl {
void readPathReferenceTypes(core::ProcessContext& context, const std::string& node_id);
void parseIdType(core::ProcessContext& context, const core::PropertyReference& prop);
- std::shared_ptr<core::logging::Logger> logger_;
-
std::string node_id_;
int32_t namespace_idx_ = 0;
opc::OPCNodeIDType id_type_{};
diff --git a/extensions/opc/src/opcbase.cpp b/extensions/opc/src/opcbase.cpp
index 27c6e10f9..7d7c7101c 100644
--- a/extensions/opc/src/opcbase.cpp
+++ b/extensions/opc/src/opcbase.cpp
@@ -21,7 +21,6 @@
#include "opc.h"
#include "opcbase.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
#include "core/ProcessSession.h"
#include "core/Core.h"
#include "utils/ProcessorConfigUtils.h"
diff --git a/extensions/opc/src/putopc.cpp b/extensions/opc/src/putopc.cpp
index c99ae531f..72a309162 100644
--- a/extensions/opc/src/putopc.cpp
+++ b/extensions/opc/src/putopc.cpp
@@ -278,12 +278,12 @@ void PutOPCProcessor::onTrigger(core::ProcessContext& context, core::ProcessSess
if (!reconnect()) {
logger_->log_warn("Could not connect to OPC server, yielding");
- yield();
+ context.yield();
return;
}
if (!readParentNodeId()) {
- yield();
+ context.yield();
return;
}
diff --git a/extensions/opc/tests/FetchOpcProcessorTests.cpp b/extensions/opc/tests/FetchOpcProcessorTests.cpp
index 247c3dad1..52b7b7533 100644
--- a/extensions/opc/tests/FetchOpcProcessorTests.cpp
+++ b/extensions/opc/tests/FetchOpcProcessorTests.cpp
@@ -26,7 +26,7 @@ namespace org::apache::nifi::minifi::test {
TEST_CASE("Test fetching using path node id", "[fetchopcprocessor]") {
OpcUaTestServer server(4841);
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto fetch_opc_processor = controller.getProcessor();
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::NodeIDType.name, "Path"));
@@ -62,7 +62,7 @@ TEST_CASE("Test fetching using path node id", "[fetchopcprocessor]") {
TEST_CASE("Test fetching using custom reference type id path", "[fetchopcprocessor]") {
OpcUaTestServer server(4841);
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto fetch_opc_processor = controller.getProcessor();
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::NodeIDType.name, "Path"));
@@ -94,7 +94,7 @@ TEST_CASE("Test fetching using custom reference type id path", "[fetchopcprocess
}
TEST_CASE("Test missing path reference types", "[fetchopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto fetch_opc_processor = controller.getProcessor();
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::NodeIDType.name, "Path"));
@@ -104,7 +104,7 @@ TEST_CASE("Test missing path reference types", "[fetchopcprocessor]") {
}
TEST_CASE("Test username and password should both be provided", "[fetchopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
REQUIRE(put_opc_processor->setProperty(processors::FetchOPCProcessor::Username.name, "user"));
@@ -114,7 +114,7 @@ TEST_CASE("Test username and password should both be provided", "[fetchopcproces
}
TEST_CASE("Test certificate path and key path should both be provided", "[fetchopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
REQUIRE(put_opc_processor->setProperty(processors::FetchOPCProcessor::CertificatePath.name, "cert"));
@@ -124,7 +124,7 @@ TEST_CASE("Test certificate path and key path should both be provided", "[fetcho
}
TEST_CASE("Test application uri should be provided if certificate is provided", "[fetchopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
REQUIRE(put_opc_processor->setProperty(processors::FetchOPCProcessor::CertificatePath.name, "cert"));
@@ -134,7 +134,7 @@ TEST_CASE("Test application uri should be provided if certificate is provided",
}
TEST_CASE("Test certificate path must be valid", "[fetchopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
REQUIRE(put_opc_processor->setProperty(processors::FetchOPCProcessor::CertificatePath.name, "/invalid/cert/path"));
@@ -145,7 +145,7 @@ TEST_CASE("Test certificate path must be valid", "[fetchopcprocessor]") {
}
TEST_CASE("Test key path must be valid", "[fetchopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
auto test_cert_path = controller.createTempDirectory() / "test_cert.pem";
{
@@ -162,7 +162,7 @@ TEST_CASE("Test key path must be valid", "[fetchopcprocessor]") {
}
TEST_CASE("Test trusted certs path must be valid", "[fetchopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
auto test_cert_path = controller.createTempDirectory() / "test_cert.pem";
{
diff --git a/extensions/opc/tests/PutOpcProcessorTests.cpp b/extensions/opc/tests/PutOpcProcessorTests.cpp
index a03ef722e..dbba5c7a7 100644
--- a/extensions/opc/tests/PutOpcProcessorTests.cpp
+++ b/extensions/opc/tests/PutOpcProcessorTests.cpp
@@ -78,7 +78,7 @@ void verifyCreatedNode(const NodeData& expected_node, SingleProcessorTestControl
TEST_CASE("Test creating a new node with path node id", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
NodeData expected_node{42, server.getNamespaceIndex(), 9999, "everything", "Simulator/Default/Device1", {}};
@@ -103,7 +103,7 @@ TEST_CASE("Test creating a new node with path node id", "[putopcprocessor]") {
TEST_CASE("Test fetching using custom reference type id path", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
NodeData expected_node{42, server.getNamespaceIndex(), 9999, "everything", "Simulator/Default/Device1/INT3/INT4", "Organizes/Organizes/HasComponent/HasComponent"};
@@ -129,7 +129,7 @@ TEST_CASE("Test fetching using custom reference type id path", "[putopcprocessor
TEST_CASE("Test fetching using custom target reference type id", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
NodeData expected_node{42, server.getNamespaceIndex(), 9999, "everything", "Simulator/Default/Device1/INT3", "Organizes/Organizes/HasComponent", "Organizes"};
@@ -154,7 +154,7 @@ TEST_CASE("Test fetching using custom target reference type id", "[putopcprocess
}
TEST_CASE("Test missing path reference types", "[putopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Path"));
@@ -173,7 +173,7 @@ TEST_CASE("Test namespace cannot be empty", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
LogTestController::getInstance().setTrace<processors::PutOPCProcessor>();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Path"));
@@ -196,7 +196,7 @@ TEST_CASE("Test valid namespace being required", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
LogTestController::getInstance().setTrace<processors::PutOPCProcessor>();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Path"));
@@ -216,7 +216,7 @@ TEST_CASE("Test valid namespace being required", "[putopcprocessor]") {
}
TEST_CASE("Test username and password should both be provided", "[putopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::Username.name, "user"));
@@ -226,7 +226,7 @@ TEST_CASE("Test username and password should both be provided", "[putopcprocesso
}
TEST_CASE("Test certificate path and key path should both be provided", "[putopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::CertificatePath.name, "cert"));
@@ -236,7 +236,7 @@ TEST_CASE("Test certificate path and key path should both be provided", "[putopc
}
TEST_CASE("Test application uri should be provided if certificate is provided", "[putopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::CertificatePath.name, "cert"));
@@ -246,7 +246,7 @@ TEST_CASE("Test application uri should be provided if certificate is provided",
}
TEST_CASE("Test certificate path must be valid", "[putopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::CertificatePath.name, "/invalid/cert/path"));
@@ -257,7 +257,7 @@ TEST_CASE("Test certificate path must be valid", "[putopcprocessor]") {
}
TEST_CASE("Test key path must be valid", "[putopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
auto test_cert_path = controller.createTempDirectory() / "test_cert.pem";
{
@@ -274,7 +274,7 @@ TEST_CASE("Test key path must be valid", "[putopcprocessor]") {
}
TEST_CASE("Test trusted certs path must be valid", "[putopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
auto test_cert_path = controller.createTempDirectory() / "test_cert.pem";
{
@@ -292,7 +292,7 @@ TEST_CASE("Test trusted certs path must be valid", "[putopcprocessor]") {
}
TEST_CASE("Test invalid int node id", "[putopcprocessor]") {
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Int"));
@@ -305,7 +305,7 @@ TEST_CASE("Test invalid int node id", "[putopcprocessor]") {
TEST_CASE("Test invalid parent node id path", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Path"));
@@ -326,7 +326,7 @@ TEST_CASE("Test invalid parent node id path", "[putopcprocessor]") {
TEST_CASE("Test missing target node id", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Path"));
@@ -349,7 +349,7 @@ TEST_CASE("Test missing target node id", "[putopcprocessor]") {
TEST_CASE("Test invalid target node id", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Path"));
@@ -372,7 +372,7 @@ TEST_CASE("Test invalid target node id", "[putopcprocessor]") {
TEST_CASE("Test missing target node type", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Path"));
@@ -395,7 +395,7 @@ TEST_CASE("Test missing target node type", "[putopcprocessor]") {
TEST_CASE("Test value type mismatch", "[putopcprocessor]") {
OpcUaTestServer server;
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::PutOPCProcessor>("PutOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::PutOPCProcessor>("PutOPCProcessor")};
auto put_opc_processor = controller.getProcessor();
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4840/"));
REQUIRE(put_opc_processor->setProperty(processors::PutOPCProcessor::ParentNodeIDType.name, "Path"));
diff --git a/extensions/python/PythonCreator.h b/extensions/python/PythonCreator.h
index 647f3255a..3cc2c1284 100644
--- a/extensions/python/PythonCreator.h
+++ b/extensions/python/PythonCreator.h
@@ -46,18 +46,18 @@ class DummyProcessorDescriptor : public core::ProcessorDescriptor {
public:
~DummyProcessorDescriptor() override = default;
- void setSupportedRelationships(std::span<const core::RelationshipDefinition> relationships) override {}
- void setSupportedProperties(std::span<const core::PropertyReference> properties) override {}
+ void setSupportedRelationships(std::span<const core::RelationshipDefinition> /*relationships*/) override {}
+ void setSupportedProperties(std::span<const core::PropertyReference> /*properties*/) override {}
};
class DummyLogger : public core::logging::Logger {
public:
- void set_max_log_size(int size) override {}
+ void set_max_log_size(int /*size*/) override {}
std::optional<std::string> get_id() override {
return std::nullopt;
}
- void log_string(core::logging::LOG_LEVEL level, std::string str) override {}
- bool should_log(core::logging::LOG_LEVEL level) override {
+ void log_string(core::logging::LOG_LEVEL /*level*/, std::string /*str*/) override {}
+ bool should_log(core::logging::LOG_LEVEL /*level*/) override {
return false;
}
[[nodiscard]] virtual core::logging::LOG_LEVEL level() const override {
diff --git a/extensions/standard-processors/tests/integration/InvokeHTTPTests.cpp b/extensions/standard-processors/tests/integration/InvokeHTTPTests.cpp
index de591b2ae..29e03ed83 100644
--- a/extensions/standard-processors/tests/integration/InvokeHTTPTests.cpp
+++ b/extensions/standard-processors/tests/integration/InvokeHTTPTests.cpp
@@ -410,8 +410,8 @@ TEST_CASE("InvokeHTTP: invalid characters are removed from outgoing HTTP headers
test_attr_value_out = test_attr_value_in;
};
- SingleProcessorTestController controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
- auto* const invoke_http = controller.getProcessor<InvokeHTTP>();
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
+ auto const invoke_http = controller.getProcessor<InvokeHTTP>();
const TestHTTPServer http_server;
REQUIRE(invoke_http->setProperty(InvokeHTTP::Method.name, "POST"));
REQUIRE(invoke_http->setProperty(InvokeHTTP::URL.name, TestHTTPServer::URL));
diff --git a/libminifi/src/core/ProcessContext.cpp b/libminifi/src/core/ProcessContext.cpp
index dd9d55099..3e453d58c 100644
--- a/libminifi/src/core/ProcessContext.cpp
+++ b/libminifi/src/core/ProcessContext.cpp
@@ -41,36 +41,33 @@ class StandardProcessorInfo : public ProcessorInfo {
ProcessContextImpl::ProcessContextImpl(
Processor& processor, controller::ControllerServiceProvider* controller_service_provider, const std::shared_ptr<core::Repository>& repo,
const std::shared_ptr<core::Repository>& flow_repo, const std::shared_ptr<core::ContentRepository>& content_repo)
- : VariableRegistryImpl(Configure::create()),
+ : VariableRegistryImpl(static_cast<std::shared_ptr<Configure>>(minifi::Configure::create())),
+ logger_(logging::LoggerFactory<ProcessContext>::getLogger()),
controller_service_provider_(controller_service_provider),
+ state_storage_(getStateStorage(logger_, controller_service_provider_, nullptr)),
+ repo_(repo),
flow_repo_(flow_repo),
content_repo_(content_repo),
processor_(processor),
- logger_(logging::LoggerFactory<ProcessContext>::getLogger()),
configure_(minifi::Configure::create()),
- initialized_(false) {
- repo_ = repo;
- state_storage_ = getStateStorage(logger_, controller_service_provider_, nullptr);
- info_ = std::make_unique<StandardProcessorInfo>(processor);
-}
+ info_(std::make_unique<StandardProcessorInfo>(processor)),
+ initialized_(false) {}
ProcessContextImpl::ProcessContextImpl(
Processor& processor, controller::ControllerServiceProvider* controller_service_provider, const std::shared_ptr<core::Repository>& repo,
const std::shared_ptr<core::Repository>& flow_repo, const std::shared_ptr<minifi::Configure>& configuration,
const std::shared_ptr<core::ContentRepository>& content_repo)
: VariableRegistryImpl(configuration),
+ logger_(logging::LoggerFactory<ProcessContext>::getLogger()),
controller_service_provider_(controller_service_provider),
+ state_storage_(getStateStorage(logger_, controller_service_provider_, configuration)),
+ repo_(repo),
flow_repo_(flow_repo),
content_repo_(content_repo),
processor_(processor),
- logger_(logging::LoggerFactory<ProcessContext>::getLogger()),
- configure_(configuration),
- initialized_(false) {
- repo_ = repo;
- state_storage_ = getStateStorage(logger_, controller_service_provider_, configuration);
- if (!configure_) { configure_ = minifi::Configure::create(); }
- info_ = std::make_unique<StandardProcessorInfo>(processor);
-}
+ configure_(configuration ? gsl::make_not_null(configuration) : minifi::Configure::create()),
+ info_(std::make_unique<StandardProcessorInfo>(processor)),
+ initialized_(false) {}
bool ProcessContextImpl::hasNonEmptyProperty(std::string_view name) const {
auto val = getProcessor().getProperty(name);
(nifi-minifi-cpp) 24/37: Windows proc update
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 3c3264d65e520678d2ea117e84ca9734ea047049
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Tue Jun 24 15:14:38 2025 +0200
Windows proc update
---
extensions/smb/tests/FetchSmbTests.cpp | 5 +++--
extensions/smb/tests/ListAndFetchSmbTests.cpp | 8 ++++----
extensions/smb/tests/ListSmbTests.cpp | 3 ++-
extensions/smb/tests/PutSmbTests.cpp | 7 ++++---
4 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/extensions/smb/tests/FetchSmbTests.cpp b/extensions/smb/tests/FetchSmbTests.cpp
index 0fe708167..921b92341 100644
--- a/extensions/smb/tests/FetchSmbTests.cpp
+++ b/extensions/smb/tests/FetchSmbTests.cpp
@@ -24,13 +24,14 @@
#include "unit/SingleProcessorTestController.h"
#include "utils/OsUtils.h"
#include "core/Resource.h"
+#include "unit/ProcessorUtils.h"
namespace org::apache::nifi::minifi::extensions::smb::test {
REGISTER_RESOURCE(MockSmbConnectionControllerService, ControllerService);
TEST_CASE("FetchSmb invalid network path") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<FetchSmb>("FetchSmb")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<FetchSmb>("FetchSmb")};
const auto fetch_smb = controller.getProcessor();
auto smb_connection_node = controller.plan->addController("MockSmbConnectionControllerService", "smb_connection_controller_service");
REQUIRE(controller.plan->setProperty(smb_connection_node, SmbConnectionControllerService::Hostname, utils::OsUtils::getHostName().value_or("localhost")));
@@ -43,7 +44,7 @@ TEST_CASE("FetchSmb invalid network path") {
}
TEST_CASE("FetchSmb tests") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<FetchSmb>("FetchSmb")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<FetchSmb>("FetchSmb")};
const auto fetch_smb = controller.getProcessor();
auto smb_connection_node = controller.plan->addController("MockSmbConnectionControllerService", "smb_connection_controller_service");
diff --git a/extensions/smb/tests/ListAndFetchSmbTests.cpp b/extensions/smb/tests/ListAndFetchSmbTests.cpp
index ee820c9e7..d413ece1b 100644
--- a/extensions/smb/tests/ListAndFetchSmbTests.cpp
+++ b/extensions/smb/tests/ListAndFetchSmbTests.cpp
@@ -33,10 +33,10 @@ using minifi::processors::ReadFromFlowFileTestProcessor;
TEST_CASE("ListSmb and FetchSmb work together") {
TestController controller;
auto plan = controller.createPlan();
- auto list_smb = dynamic_cast<ListSmb*>(plan->addProcessor("ListSmb", "list_smb"));
- auto fetch_smb = dynamic_cast<FetchSmb*>(plan->addProcessor("FetchSmb", "fetch_smb"));
- auto read_from_success_relationship = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_success_relationship"));
- auto read_from_failure_relationship = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_failure_relationship"));
+ auto list_smb = plan->addProcessor<ListSmb>("list_smb");
+ auto fetch_smb = plan->addProcessor<ListSmb>("fetch_smb");
+ auto read_from_success_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_success_relationship");
+ auto read_from_failure_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_failure_relationship");
plan->addConnection(list_smb, ListSmb::Success, fetch_smb);
diff --git a/extensions/smb/tests/ListSmbTests.cpp b/extensions/smb/tests/ListSmbTests.cpp
index 15120d029..d7b197831 100644
--- a/extensions/smb/tests/ListSmbTests.cpp
+++ b/extensions/smb/tests/ListSmbTests.cpp
@@ -25,13 +25,14 @@
#include "range/v3/algorithm/count_if.hpp"
#include "range/v3/algorithm/find_if.hpp"
#include "core/Resource.h"
+#include "unit/ProcessorUtils.h"
namespace org::apache::nifi::minifi::extensions::smb::test {
REGISTER_RESOURCE(MockSmbConnectionControllerService, ControllerService);
TEST_CASE("ListSmb invalid network path") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ListSmb>("ListSmb")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ListSmb>("ListSmb")};
const auto list_smb = controller.getProcessor();
auto smb_connection_node = controller.plan->addController("MockSmbConnectionControllerService", "smb_connection_controller_service");
REQUIRE(controller.plan->setProperty(smb_connection_node, SmbConnectionControllerService::Hostname, utils::OsUtils::getHostName().value_or("localhost")));
diff --git a/extensions/smb/tests/PutSmbTests.cpp b/extensions/smb/tests/PutSmbTests.cpp
index a60f99e43..1f86d83e2 100644
--- a/extensions/smb/tests/PutSmbTests.cpp
+++ b/extensions/smb/tests/PutSmbTests.cpp
@@ -23,6 +23,7 @@
#include "unit/SingleProcessorTestController.h"
#include "range/v3/algorithm/count_if.hpp"
#include "core/Resource.h"
+#include "unit/ProcessorUtils.h"
namespace org::apache::nifi::minifi::extensions::smb::test {
@@ -35,7 +36,7 @@ std::string checkFileContent(const std::filesystem::path& path) {
}
TEST_CASE("PutSmb invalid network path") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<PutSmb>("PutSmb")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<PutSmb>("PutSmb")};
const auto put_smb = controller.getProcessor();
auto smb_connection_node = controller.plan->addController("MockSmbConnectionControllerService", "smb_connection_controller_service");
REQUIRE(controller.plan->setProperty(smb_connection_node, SmbConnectionControllerService::Hostname, utils::OsUtils::getHostName().value_or("localhost")));
@@ -48,7 +49,7 @@ TEST_CASE("PutSmb invalid network path") {
}
TEST_CASE("PutSmb conflict resolution test") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<PutSmb>("PutSmb")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<PutSmb>("PutSmb")};
const auto put_smb = controller.getProcessor();
auto temp_directory = controller.createTempDirectory();
@@ -133,7 +134,7 @@ TEST_CASE("PutSmb conflict resolution test") {
}
TEST_CASE("PutSmb create missing dirs test") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<PutSmb>("PutSmb")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<PutSmb>("PutSmb")};
const auto put_smb = controller.getProcessor();
auto temp_directory = controller.createTempDirectory();
(nifi-minifi-cpp) 36/37: Fix parameter name
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 567aa7606112e13bcc11e1f098b6b98a82861465
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Fri Aug 8 13:44:49 2025 +0200
Fix parameter name
---
libminifi/include/core/FlowConfiguration.h | 2 +-
libminifi/include/processors/ProcessorUtils.h | 12 +++---------
libminifi/src/core/FlowConfiguration.cpp | 4 ++--
3 files changed, 6 insertions(+), 12 deletions(-)
diff --git a/libminifi/include/core/FlowConfiguration.h b/libminifi/include/core/FlowConfiguration.h
index 0c9cd327c..652674651 100644
--- a/libminifi/include/core/FlowConfiguration.h
+++ b/libminifi/include/core/FlowConfiguration.h
@@ -84,7 +84,7 @@ class FlowConfiguration : public CoreComponentImpl {
~FlowConfiguration() override;
// Create Processor (Node/Input/Output Port) based on the name
- std::unique_ptr<core::Processor> createProcessor(const std::string &class_short, const std::string &fullclass, const std::string &name, const utils::Identifier &uuid);
+ std::unique_ptr<core::Processor> createProcessor(const std::string &class_short, const std::string &fullclass, const std::string &object_name, const utils::Identifier &uuid);
// Create Root Processor Group
static std::unique_ptr<core::ProcessGroup> createRootProcessGroup(const std::string &name, const utils::Identifier &uuid, int version);
diff --git a/libminifi/include/processors/ProcessorUtils.h b/libminifi/include/processors/ProcessorUtils.h
index 7f5452e78..6b750ac8e 100644
--- a/libminifi/include/processors/ProcessorUtils.h
+++ b/libminifi/include/processors/ProcessorUtils.h
@@ -30,16 +30,10 @@ class ProcessorUtils {
public:
ProcessorUtils() = delete;
- /**
- * Instantiates and configures a processor
- * @param class_short_name short name of the class
- * @param canonicalName full class name ( canonical name )
- * @param uuid uuid object for the processor
- */
- static inline std::unique_ptr<core::Processor> createProcessor(const std::string &class_short_name, const std::string &canonicalName, const std::string &name, const utils::Identifier &uuid) {
- auto ptr = core::ClassLoader::getDefaultClassLoader().instantiate(class_short_name, name, uuid);
+ static inline std::unique_ptr<core::Processor> createProcessor(const std::string &class_short, const std::string &fullclass, const std::string &object_name, const utils::Identifier &uuid) {
+ auto ptr = core::ClassLoader::getDefaultClassLoader().instantiate(class_short, object_name, uuid);
if (ptr == nullptr) {
- ptr = core::ClassLoader::getDefaultClassLoader().instantiate(canonicalName, name, uuid);
+ ptr = core::ClassLoader::getDefaultClassLoader().instantiate(fullclass, object_name, uuid);
}
if (ptr == nullptr) {
return nullptr;
diff --git a/libminifi/src/core/FlowConfiguration.cpp b/libminifi/src/core/FlowConfiguration.cpp
index 379c18fe3..ed4041e3f 100644
--- a/libminifi/src/core/FlowConfiguration.cpp
+++ b/libminifi/src/core/FlowConfiguration.cpp
@@ -71,8 +71,8 @@ FlowConfiguration::~FlowConfiguration() {
}
}
-std::unique_ptr<core::Processor> FlowConfiguration::createProcessor(const std::string &class_short, const std::string &fullclass, const std::string &name, const utils::Identifier &uuid) {
- auto processor = minifi::processors::ProcessorUtils::createProcessor(class_short, fullclass, name, uuid);
+std::unique_ptr<core::Processor> FlowConfiguration::createProcessor(const std::string &class_short, const std::string &fullclass, const std::string &object_name, const utils::Identifier &uuid) {
+ auto processor = minifi::processors::ProcessorUtils::createProcessor(class_short, fullclass, object_name, uuid);
if (nullptr == processor) {
logger_->log_error("No Processor defined for {}", fullclass);
return nullptr;
(nifi-minifi-cpp) 22/37: Update extensions/python/ExecutePythonProcessor.cpp
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 6dbbb6cfd6c08b0443406516749f02a3ef601bb0
Author: Adam Debreceni <64...@users.noreply.github.com>
AuthorDate: Tue Jun 24 10:38:51 2025 +0200
Update extensions/python/ExecutePythonProcessor.cpp
Co-authored-by: Márton Szász <sz...@apache.org>
---
extensions/python/ExecutePythonProcessor.cpp | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/extensions/python/ExecutePythonProcessor.cpp b/extensions/python/ExecutePythonProcessor.cpp
index bdc7f6bbb..7daec15c1 100644
--- a/extensions/python/ExecutePythonProcessor.cpp
+++ b/extensions/python/ExecutePythonProcessor.cpp
@@ -57,9 +57,7 @@ void ExecutePythonProcessor::initializeScript() {
void ExecutePythonProcessor::initialize() {
initializeScript();
std::vector<core::PropertyReference> all_properties{Properties.begin(), Properties.end()};
- for (auto& python_prop : python_properties_) {
- all_properties.push_back(python_prop.getReference());
- }
+ ranges::transform(python_properties_, std::back_inserter(all_properties), &core::Property::getReference);
setSupportedProperties(all_properties);
setSupportedRelationships(Relationships);
logger_->log_debug("Processor has already been initialized, returning...");
(nifi-minifi-cpp) 37/37: Fix rebase
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 75cdd49668013d5864cdbe1edffb3d8ca29e2577
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Aug 11 16:23:26 2025 +0200
Fix rebase
---
extension-utils/include/controllers/SSLContextService.h | 0
extensions/opc/tests/FetchOpcProcessorTests.cpp | 9 +++++----
extensions/standard-processors/processors/PutTCP.h | 1 -
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/extension-utils/include/controllers/SSLContextService.h b/extension-utils/include/controllers/SSLContextService.h
deleted file mode 100644
index e69de29bb..000000000
diff --git a/extensions/opc/tests/FetchOpcProcessorTests.cpp b/extensions/opc/tests/FetchOpcProcessorTests.cpp
index 52b7b7533..09da30ae0 100644
--- a/extensions/opc/tests/FetchOpcProcessorTests.cpp
+++ b/extensions/opc/tests/FetchOpcProcessorTests.cpp
@@ -20,6 +20,7 @@
#include "OpcUaTestServer.h"
#include "unit/SingleProcessorTestController.h"
#include "include/fetchopc.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::test {
@@ -182,7 +183,7 @@ TEST_CASE("Test trusted certs path must be valid", "[fetchopcprocessor]") {
TEST_CASE("Test no fetch result using lazy mode when no timestamps are changed", "[fetchopcprocessor]") {
OpcUaTestServer server(4841);
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
LogTestController::getInstance().setDebug<processors::FetchOPCProcessor>();
auto fetch_opc_processor = controller.getProcessor();
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
@@ -204,7 +205,7 @@ TEST_CASE("Test no fetch result using lazy mode when no timestamps are changed",
TEST_CASE("Test fetch for nodes with changed timestamps with lazy mode", "[fetchopcprocessor]") {
OpcUaTestServer server(4841);
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
LogTestController::getInstance().setDebug<processors::FetchOPCProcessor>();
auto fetch_opc_processor = controller.getProcessor();
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
@@ -228,7 +229,7 @@ TEST_CASE("Test fetch for nodes with changed timestamps with lazy mode", "[fetch
TEST_CASE("Test no fetch result using lazy new value mode when no values are changed", "[fetchopcprocessor]") {
OpcUaTestServer server(4841);
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
LogTestController::getInstance().setDebug<processors::FetchOPCProcessor>();
auto fetch_opc_processor = controller.getProcessor();
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
@@ -251,7 +252,7 @@ TEST_CASE("Test no fetch result using lazy new value mode when no values are cha
TEST_CASE("Test fetching new values using lazy new value mode", "[fetchopcprocessor]") {
OpcUaTestServer server(4841);
server.start();
- SingleProcessorTestController controller{std::make_unique<processors::FetchOPCProcessor>("FetchOPCProcessor")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::FetchOPCProcessor>("FetchOPCProcessor")};
LogTestController::getInstance().setDebug<processors::FetchOPCProcessor>();
auto fetch_opc_processor = controller.getProcessor();
REQUIRE(fetch_opc_processor->setProperty(processors::FetchOPCProcessor::OPCServerEndPoint.name, "opc.tcp://127.0.0.1:4841/"));
diff --git a/extensions/standard-processors/processors/PutTCP.h b/extensions/standard-processors/processors/PutTCP.h
index f95ec64c3..abad519d2 100644
--- a/extensions/standard-processors/processors/PutTCP.h
+++ b/extensions/standard-processors/processors/PutTCP.h
@@ -32,7 +32,6 @@
#include "asio/ssl/context.hpp"
#include "controllers/SSLContextServiceInterface.h"
#include "core/Core.h"
-#include "core/Processor.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/RelationshipDefinition.h"
(nifi-minifi-cpp) 29/37: Fix linux build
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit c236217bc666efff064304abbae62c2fc066b7b9
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Jul 7 11:50:47 2025 +0200
Fix linux build
---
extensions/systemd/ConsumeJournald.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/extensions/systemd/ConsumeJournald.cpp b/extensions/systemd/ConsumeJournald.cpp
index 0a7043ed9..3afbbd899 100644
--- a/extensions/systemd/ConsumeJournald.cpp
+++ b/extensions/systemd/ConsumeJournald.cpp
@@ -34,7 +34,7 @@ namespace org::apache::nifi::minifi::extensions::systemd {
namespace chr = std::chrono;
ConsumeJournald::ConsumeJournald(core::ProcessorMetadata metadata, std::unique_ptr<libwrapper::LibWrapper>&& libwrapper)
- : core::ProcessorImpl{info}, libwrapper_{std::move(libwrapper)} {}
+ : core::ProcessorImpl{metadata}, libwrapper_{std::move(libwrapper)} {}
void ConsumeJournald::initialize() {
setSupportedProperties(Properties);
(nifi-minifi-cpp) 20/37: Add dtor, review changes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit fc75710b3a4441190d4ed472a604fcc74f24a4df
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Jun 23 13:04:19 2025 +0200
Add dtor, review changes
---
extensions/expression-language/ProcessContextExpr.h | 2 +-
libminifi/include/ForwardingNode.h | 2 ++
libminifi/include/Port.h | 4 ++++
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/extensions/expression-language/ProcessContextExpr.h b/extensions/expression-language/ProcessContextExpr.h
index 228b1e35f..3aa822d83 100644
--- a/extensions/expression-language/ProcessContextExpr.h
+++ b/extensions/expression-language/ProcessContextExpr.h
@@ -39,7 +39,7 @@ class ProcessContextExpr final : public core::ProcessContextImpl {
nonstd::expected<std::string, std::error_code> getProperty(std::string_view name, const FlowFile*) const override;
nonstd::expected<std::string, std::error_code> getDynamicProperty(std::string_view name, const FlowFile*) const override;
-//
+
nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value) override;
nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value) override;
diff --git a/libminifi/include/ForwardingNode.h b/libminifi/include/ForwardingNode.h
index 914d62e43..fb173debf 100644
--- a/libminifi/include/ForwardingNode.h
+++ b/libminifi/include/ForwardingNode.h
@@ -41,6 +41,8 @@ class ForwardingNode : public core::ProcessorImpl {
void initialize() override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
+
+ ~ForwardingNode() override = default;
};
} // namespace org::apache::nifi::minifi
diff --git a/libminifi/include/Port.h b/libminifi/include/Port.h
index 0cfd06133..8f1e20caf 100644
--- a/libminifi/include/Port.h
+++ b/libminifi/include/Port.h
@@ -41,6 +41,8 @@ class PortImpl final : public ForwardingNode {
return port_type_;
}
+ ~PortImpl() override = default;
+
MINIFIAPI static constexpr core::annotation::Input InputRequirement = core::annotation::Input::INPUT_ALLOWED;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
@@ -57,6 +59,8 @@ class Port : public core::Processor {
gsl_Assert(port_impl);
return port_impl->getPortType();
}
+
+ ~Port() override = default;
};
} // namespace org::apache::nifi::minifi
(nifi-minifi-cpp) 02/37: Tests work
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit b0fb553b91a2e20803faaaad7959c78b36de9106
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Mar 6 10:50:25 2025 +0100
Tests work
---
extension-utils/include/core/AbstractProcessor.h | 2 +-
extensions/aws/processors/DeleteS3Object.h | 8 +-
extensions/aws/processors/FetchS3Object.h | 8 +-
extensions/aws/processors/ListS3.h | 8 +-
extensions/aws/processors/PutS3Object.h | 8 +-
extensions/aws/processors/S3Processor.cpp | 9 +-
extensions/aws/processors/S3Processor.h | 5 +-
extensions/aws/tests/PutS3ObjectTests.cpp | 4 +-
extensions/aws/tests/S3TestsFixture.h | 8 +-
.../processors/AzureBlobStorageProcessorBase.cpp | 12 +-
.../processors/AzureBlobStorageProcessorBase.h | 6 +-
.../AzureBlobStorageSingleBlobProcessorBase.h | 6 +-
.../AzureDataLakeStorageFileProcessorBase.h | 9 +-
.../processors/AzureDataLakeStorageProcessorBase.h | 10 +-
.../azure/processors/AzureStorageProcessorBase.h | 7 +-
.../azure/processors/DeleteAzureBlobStorage.h | 8 +-
.../azure/processors/DeleteAzureDataLakeStorage.h | 8 +-
.../azure/processors/FetchAzureBlobStorage.h | 8 +-
.../azure/processors/FetchAzureDataLakeStorage.h | 8 +-
extensions/azure/processors/ListAzureBlobStorage.h | 8 +-
.../azure/processors/ListAzureDataLakeStorage.h | 8 +-
extensions/azure/processors/PutAzureBlobStorage.h | 8 +-
.../azure/processors/PutAzureDataLakeStorage.h | 8 +-
.../azure/tests/AzureBlobStorageTestsFixture.h | 7 +-
.../azure/tests/AzureDataLakeStorageTestsFixture.h | 6 +-
.../azure/tests/ListAzureBlobStorageTests.cpp | 4 +-
.../azure/tests/ListAzureDataLakeStorageTests.cpp | 6 +-
extensions/bustache/ApplyTemplate.h | 7 +-
extensions/bustache/tests/ApplyTemplateTests.cpp | 3 +-
extensions/civetweb/processors/ListenHTTP.cpp | 6 +-
extensions/civetweb/processors/ListenHTTP.h | 7 +-
extensions/civetweb/tests/C2VerifyServeResults.cpp | 6 +-
extensions/civetweb/tests/ListenHTTPTests.cpp | 17 +-
.../couchbase/tests/GetCouchbaseKeyTests.cpp | 3 +-
.../couchbase/tests/PutCouchbaseKeyTests.cpp | 3 +-
extensions/elasticsearch/PostElasticsearch.h | 7 +-
.../elasticsearch/tests/PostElasticsearchTests.cpp | 3 +-
extensions/execute-process/ExecuteProcess.cpp | 14 +-
extensions/execute-process/ExecuteProcess.h | 12 +-
.../execute-process/test/ExecuteProcessTests.cpp | 7 +-
.../expression-language/ProcessContextExpr.cpp | 31 ++-
.../expression-language/ProcessContextExpr.h | 11 +-
.../tests/ProcessContextExprTests.cpp | 2 +-
extensions/gcp/processors/DeleteGCSObject.h | 4 +-
extensions/gcp/processors/FetchGCSObject.h | 4 +-
extensions/gcp/processors/GCSProcessor.h | 8 +-
extensions/gcp/processors/ListGCSBucket.h | 4 +-
extensions/gcp/processors/PutGCSObject.h | 4 +-
extensions/gcp/tests/DeleteGCSObjectTests.cpp | 17 +-
extensions/gcp/tests/FetchGCSObjectTests.cpp | 15 +-
extensions/gcp/tests/ListGCSBucketTests.cpp | 17 +-
extensions/gcp/tests/PutGCSObjectTests.cpp | 47 ++--
extensions/grafana-loki/PushGrafanaLoki.h | 10 +-
extensions/grafana-loki/PushGrafanaLokiGrpc.h | 4 +-
extensions/grafana-loki/PushGrafanaLokiREST.h | 4 +-
.../grafana-loki/tests/PushGrafanaLokiGrpcTest.cpp | 12 +-
.../grafana-loki/tests/PushGrafanaLokiRESTTest.cpp | 14 +-
extensions/kafka/ConsumeKafka.h | 3 +-
extensions/kafka/KafkaProcessorBase.h | 8 +-
extensions/kafka/PublishKafka.h | 3 +-
extensions/kafka/tests/PublishKafkaTests.cpp | 17 +-
extensions/libarchive/BinFiles.cpp | 2 +-
extensions/libarchive/CompressContent.h | 5 +-
extensions/libarchive/FocusArchiveEntry.cpp | 16 +-
extensions/libarchive/FocusArchiveEntry.h | 11 +-
extensions/libarchive/ManipulateArchive.cpp | 4 +-
extensions/libarchive/ManipulateArchive.h | 7 +-
extensions/libarchive/MergeContent.h | 5 +-
extensions/libarchive/UnfocusArchiveEntry.h | 5 +-
.../libarchive/tests/CompressContentTests.cpp | 2 +-
extensions/libarchive/tests/FocusArchiveTests.cpp | 5 +-
.../libarchive/tests/ManipulateArchiveTests.cpp | 3 +-
extensions/libarchive/tests/MergeFileTests.cpp | 8 +-
.../TestExecuteScriptProcessorWithLuaScript.cpp | 23 +-
.../mqtt/processors/AbstractMQTTProcessor.cpp | 4 +-
extensions/mqtt/processors/AbstractMQTTProcessor.h | 8 +-
extensions/mqtt/processors/ConsumeMQTT.cpp | 6 +-
extensions/mqtt/processors/ConsumeMQTT.h | 7 +-
extensions/mqtt/processors/PublishMQTT.cpp | 10 +-
extensions/mqtt/processors/PublishMQTT.h | 9 +-
extensions/opc/include/fetchopc.h | 5 +-
extensions/opc/include/opcbase.h | 5 +-
extensions/opc/include/putopc.h | 5 +-
extensions/opc/src/fetchopc.cpp | 10 +-
extensions/opencv/CaptureRTSPFrame.h | 7 +-
extensions/opencv/MotionDetector.h | 7 +-
extensions/python/ExecutePythonProcessor.cpp | 30 +--
extensions/python/ExecutePythonProcessor.h | 18 +-
extensions/python/PythonCreator.h | 63 ++++-
extensions/python/PythonObjectFactory.h | 59 +----
extensions/python/PythonProcessor.cpp | 4 +-
extensions/python/PythonProcessor.h | 2 +-
extensions/python/PythonScriptEngine.cpp | 4 +-
extensions/python/PythonScriptEngine.h | 8 +-
.../TestExecuteScriptProcessorWithPythonScript.cpp | 23 +-
extensions/rocksdb-repos/tests/RepoTests.cpp | 2 +-
extensions/rocksdb-repos/tests/SwapTests.cpp | 8 +-
extensions/script/ExecuteScript.cpp | 2 +-
extensions/script/ExecuteScript.h | 5 +-
extensions/sftp/processors/FetchSFTP.cpp | 5 -
extensions/sftp/processors/FetchSFTP.h | 2 +-
extensions/sftp/processors/ListSFTP.cpp | 5 -
extensions/sftp/processors/ListSFTP.h | 2 +-
extensions/sftp/processors/PutSFTP.cpp | 5 +-
extensions/sftp/processors/PutSFTP.h | 2 +-
extensions/sftp/processors/SFTPProcessorBase.cpp | 4 +-
extensions/sftp/processors/SFTPProcessorBase.h | 6 +-
extensions/splunk/PutSplunkHTTP.cpp | 2 +-
extensions/splunk/PutSplunkHTTP.h | 5 +-
extensions/splunk/QuerySplunkIndexingStatus.h | 4 +-
extensions/splunk/SplunkHECProcessor.h | 7 +-
extensions/splunk/tests/PutSplunkHTTPTests.cpp | 60 ++---
.../tests/QuerySplunkIndexingStatusTests.cpp | 66 +++---
extensions/sql/processors/ExecuteSQL.cpp | 4 -
extensions/sql/processors/ExecuteSQL.h | 2 +-
extensions/sql/processors/PutSQL.cpp | 4 -
extensions/sql/processors/PutSQL.h | 2 +-
extensions/sql/processors/QueryDatabaseTable.cpp | 4 -
extensions/sql/processors/QueryDatabaseTable.h | 2 +-
extensions/sql/processors/SQLProcessor.h | 7 +-
extensions/sql/tests/PutSQLTests.cpp | 3 +-
.../standard-processors/modbus/FetchModbusTcp.cpp | 10 +-
.../standard-processors/modbus/FetchModbusTcp.h | 7 +-
.../processors/AppendHostInfo.h | 7 +-
.../processors/AttributesToJSON.h | 7 +-
.../processors/DefragmentText.h | 5 +-
.../standard-processors/processors/ExtractText.h | 2 +-
.../standard-processors/processors/FetchFile.h | 7 +-
.../processors/GenerateFlowFile.cpp | 3 +-
.../processors/GenerateFlowFile.h | 2 +-
.../standard-processors/processors/GetFile.cpp | 6 +-
.../standard-processors/processors/GetFile.h | 9 +-
extensions/standard-processors/processors/GetTCP.h | 7 +-
.../standard-processors/processors/HashContent.h | 7 +-
.../standard-processors/processors/InvokeHTTP.cpp | 4 +-
.../standard-processors/processors/InvokeHTTP.h | 9 +-
.../processors/JoltTransformJSON.h | 8 +-
.../standard-processors/processors/ListFile.h | 5 +-
.../standard-processors/processors/ListenSyslog.h | 4 +-
.../standard-processors/processors/ListenTCP.h | 4 +-
.../standard-processors/processors/ListenUDP.h | 4 +-
.../standard-processors/processors/LogAttribute.h | 8 +-
.../processors/NetworkListenerProcessor.h | 10 +-
.../standard-processors/processors/PutFile.h | 7 +-
.../standard-processors/processors/PutTCP.cpp | 11 +-
extensions/standard-processors/processors/PutTCP.h | 2 +-
.../standard-processors/processors/PutUDP.cpp | 10 +-
extensions/standard-processors/processors/PutUDP.h | 5 +-
.../standard-processors/processors/ReplaceText.cpp | 11 +-
.../standard-processors/processors/ReplaceText.h | 6 +-
.../standard-processors/processors/RetryFlowFile.h | 5 +-
.../processors/RouteOnAttribute.cpp | 1 -
.../processors/RouteOnAttribute.h | 2 +-
.../standard-processors/processors/RouteText.cpp | 9 +-
.../standard-processors/processors/RouteText.h | 6 +-
.../processors/SegmentContent.h | 2 +-
.../standard-processors/processors/SplitContent.h | 6 +-
.../standard-processors/processors/SplitRecord.cpp | 2 +-
.../standard-processors/processors/SplitText.h | 2 +-
.../standard-processors/processors/TailFile.cpp | 4 +-
.../standard-processors/processors/TailFile.h | 7 +-
.../processors/UpdateAttribute.cpp | 2 +-
.../processors/UpdateAttribute.h | 2 +-
.../tests/integration/InvokeHTTPTests.cpp | 20 +-
.../tests/unit/AttributeRollingWindowTests.cpp | 3 +-
.../tests/unit/DefragmentTextTests.cpp | 254 ++++++++++-----------
.../tests/unit/ExtractTextTests.cpp | 2 +-
.../tests/unit/FetchFileTests.cpp | 6 +-
.../tests/unit/FlowJsonTests.cpp | 32 +--
.../tests/unit/GenerateFlowFileTests.cpp | 13 +-
.../tests/unit/GetFileTests.cpp | 4 +-
.../standard-processors/tests/unit/GetTCPTests.cpp | 8 +-
.../tests/unit/HashContentTest.cpp | 4 +-
.../tests/unit/JoltTransformJSONTests.cpp | 14 +-
.../tests/unit/ListFileTests.cpp | 4 +-
.../tests/unit/ListenSyslogTests.cpp | 24 +-
.../tests/unit/ListenTcpTests.cpp | 30 +--
.../tests/unit/ListenUDPTests.cpp | 12 +-
.../tests/unit/LogAttributeTests.cpp | 8 +-
.../tests/unit/ProcessGroupTestUtils.h | 2 +-
.../tests/unit/ProcessorTests.cpp | 28 ++-
.../tests/unit/PutFileTests.cpp | 2 +-
.../standard-processors/tests/unit/PutTCPTests.cpp | 6 +-
.../standard-processors/tests/unit/PutUDPTests.cpp | 3 +-
.../tests/unit/ReplaceTextTests.cpp | 8 +-
.../tests/unit/RouteTextTests.cpp | 3 +-
.../tests/unit/SegmentContentTests.cpp | 15 +-
.../tests/unit/SplitContentTests.cpp | 49 ++--
.../tests/unit/SplitRecordTests.cpp | 3 +-
.../tests/unit/SplitTextTests.cpp | 25 +-
.../tests/unit/TailFileTests.cpp | 14 +-
.../tests/unit/YamlConfigurationTests.cpp | 6 +-
.../tests/unit/YamlConnectionParserTest.cpp | 16 +-
extensions/test-processors/KamikazeProcessor.h | 5 +-
.../test-processors/LogOnDestructionProcessor.h | 5 +-
libminifi/include/ForwardingNode.h | 6 +-
libminifi/include/Funnel.h | 3 +-
libminifi/include/Port.h | 22 +-
libminifi/include/RemoteProcessorGroupPort.h | 8 +-
libminifi/include/core/Processor.h | 12 +
libminifi/include/core/ProcessorProxy.h | 20 ++
.../reporting/SiteToSiteProvenanceReportingTask.h | 5 +-
libminifi/src/core/ConfigurableComponentImpl.cpp | 149 ++++++++++++
libminifi/src/core/Connectable.cpp | 173 ++++++++++++++
libminifi/src/core/FlowConfiguration.cpp | 4 +-
libminifi/src/core/ProcessContext.cpp | 26 +++
libminifi/src/core/Processor.cpp | 10 +-
.../src/core/flow/StructuredConfiguration.cpp | 37 +--
.../src/core/flow/StructuredConnectionParser.cpp | 3 +-
.../SiteToSiteProvenanceReportingTask.cpp | 2 +-
libminifi/test/flow-tests/CustomProcessors.h | 8 +-
libminifi/test/flow-tests/CycleTest.cpp | 14 +-
libminifi/test/flow-tests/FlowControllerTests.cpp | 40 ++--
libminifi/test/flow-tests/LoopTest.cpp | 10 +-
libminifi/test/flow-tests/MultiLoopTest.cpp | 10 +-
libminifi/test/flow-tests/SessionTests.cpp | 5 +-
libminifi/test/integration/C2NullConfiguration.cpp | 6 +-
libminifi/test/integration/C2PauseResumeTest.cpp | 5 +-
.../integration/StateTransactionalityTests.cpp | 8 +-
libminifi/test/libtest/unit/DummyProcessor.h | 13 +-
libminifi/test/libtest/unit/MockClasses.h | 19 +-
.../libtest/unit/ReadFromFlowFileTestProcessor.h | 5 +-
.../libtest/unit/SingleProcessorTestController.h | 7 +-
libminifi/test/libtest/unit/TestBase.h | 43 ++++
libminifi/test/libtest/unit/TestUtils.h | 43 +++-
.../libtest/unit/WriteToFlowFileTestProcessor.h | 7 +-
.../test/persistence-tests/PersistenceTests.cpp | 11 +-
libminifi/test/unit/AbstractProcessorTest.cpp | 11 +-
libminifi/test/unit/ComponentManifestTests.cpp | 1 +
libminifi/test/unit/MetricsTests.cpp | 8 +-
libminifi/test/unit/ProcessorConfigUtilsTests.cpp | 16 +-
libminifi/test/unit/ResponseNodeLoaderTests.cpp | 5 +-
libminifi/test/unit/SchedulingAgentTests.cpp | 29 +--
.../minifi-cpp/core/ConfigurableComponent.h | 3 +
.../include/minifi-cpp/core/ProcessContext.h | 11 +
.../include/minifi-cpp/core/ProcessContextApi.h | 23 +-
utils/include/core/ConfigurableComponentImpl.h | 4 +-
utils/include/core/ProcessContext.h | 8 +
utils/include/core/ProcessorImpl.h | 7 +-
utils/src/core/Connectable.cpp | 2 +
utils/src/core/Processor.cpp | 15 +-
241 files changed, 1584 insertions(+), 1299 deletions(-)
diff --git a/extension-utils/include/core/AbstractProcessor.h b/extension-utils/include/core/AbstractProcessor.h
index 27a0966f2..40710d205 100644
--- a/extension-utils/include/core/AbstractProcessor.h
+++ b/extension-utils/include/core/AbstractProcessor.h
@@ -22,7 +22,7 @@
#include "range/v3/view/transform.hpp"
#include "minifi-cpp/core/Annotation.h"
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "minifi-cpp/core/RelationshipDefinition.h"
#include "utils/StringUtils.h"
diff --git a/extensions/aws/processors/DeleteS3Object.h b/extensions/aws/processors/DeleteS3Object.h
index 2d983d2a8..6a2047755 100644
--- a/extensions/aws/processors/DeleteS3Object.h
+++ b/extensions/aws/processors/DeleteS3Object.h
@@ -64,9 +64,7 @@ class DeleteS3Object : public S3Processor {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit DeleteS3Object(const std::string& name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : S3Processor(name, uuid, core::logging::LoggerFactory<DeleteS3Object>::getLogger(uuid)) {
- }
+ using S3Processor::S3Processor;
~DeleteS3Object() override = default;
@@ -76,8 +74,8 @@ class DeleteS3Object : public S3Processor {
private:
friend class ::FlowProcessorS3TestsFixture<DeleteS3Object>;
- explicit DeleteS3Object(std::string_view name, const minifi::utils::Identifier& uuid, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : S3Processor(name, uuid, core::logging::LoggerFactory<DeleteS3Object>::getLogger(uuid), std::move(s3_request_sender)) {
+ explicit DeleteS3Object(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : S3Processor(info, std::move(s3_request_sender)) {
}
std::optional<aws::s3::DeleteObjectRequestParameters> buildDeleteS3RequestParams(
diff --git a/extensions/aws/processors/FetchS3Object.h b/extensions/aws/processors/FetchS3Object.h
index ea933b4e7..fa5fd182f 100644
--- a/extensions/aws/processors/FetchS3Object.h
+++ b/extensions/aws/processors/FetchS3Object.h
@@ -74,9 +74,7 @@ class FetchS3Object : public S3Processor {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit FetchS3Object(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : S3Processor(name, uuid, core::logging::LoggerFactory<FetchS3Object>::getLogger(uuid)) {
- }
+ using S3Processor::S3Processor;
~FetchS3Object() override = default;
@@ -87,8 +85,8 @@ class FetchS3Object : public S3Processor {
private:
friend class ::FlowProcessorS3TestsFixture<FetchS3Object>;
- explicit FetchS3Object(const std::string& name, const minifi::utils::Identifier& uuid, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : S3Processor(name, uuid, core::logging::LoggerFactory<FetchS3Object>::getLogger(uuid), std::move(s3_request_sender)) {
+ explicit FetchS3Object(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : S3Processor(info, std::move(s3_request_sender)) {
}
std::optional<aws::s3::GetObjectRequestParameters> buildFetchS3RequestParams(
diff --git a/extensions/aws/processors/ListS3.h b/extensions/aws/processors/ListS3.h
index 7b3bcbefb..b3e457952 100644
--- a/extensions/aws/processors/ListS3.h
+++ b/extensions/aws/processors/ListS3.h
@@ -94,11 +94,9 @@ class ListS3 : public S3Processor {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit ListS3(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : S3Processor(name, uuid, core::logging::LoggerFactory<ListS3>::getLogger(uuid)) {
- }
- explicit ListS3(const std::string& name, minifi::utils::Identifier uuid, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : S3Processor(name, uuid, core::logging::LoggerFactory<ListS3>::getLogger(uuid), std::move(s3_request_sender)) {
+ using S3Processor::S3Processor;
+ explicit ListS3(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : S3Processor(info, std::move(s3_request_sender)) {
}
~ListS3() override = default;
diff --git a/extensions/aws/processors/PutS3Object.h b/extensions/aws/processors/PutS3Object.h
index 1af4d1cfd..0a5cbfe4e 100644
--- a/extensions/aws/processors/PutS3Object.h
+++ b/extensions/aws/processors/PutS3Object.h
@@ -180,9 +180,7 @@ class PutS3Object : public S3Processor {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PutS3Object(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : S3Processor(name, uuid, core::logging::LoggerFactory<PutS3Object>::getLogger(uuid)) {
- }
+ using S3Processor::S3Processor;
~PutS3Object() override = default;
@@ -196,8 +194,8 @@ class PutS3Object : public S3Processor {
friend class ::FlowProcessorS3TestsFixture<PutS3Object>;
- explicit PutS3Object(const std::string& name, const minifi::utils::Identifier& uuid, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : S3Processor(name, uuid, core::logging::LoggerFactory<PutS3Object>::getLogger(uuid), std::move(s3_request_sender)) {
+ explicit PutS3Object(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : S3Processor(info, std::move(s3_request_sender)) {
}
virtual uint64_t getMinPartSize() const {
diff --git a/extensions/aws/processors/S3Processor.cpp b/extensions/aws/processors/S3Processor.cpp
index 6be93f626..e45c3fa1c 100644
--- a/extensions/aws/processors/S3Processor.cpp
+++ b/extensions/aws/processors/S3Processor.cpp
@@ -32,17 +32,14 @@
namespace org::apache::nifi::minifi::aws::processors {
-S3Processor::S3Processor(const std::string_view name, const minifi::utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger)
- : AwsProcessor(name, uuid, std::move(logger)) {}
-
-S3Processor::S3Processor(const std::string_view name, const minifi::utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
- : AwsProcessor(name, uuid, std::move(logger)),
+S3Processor::S3Processor(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender)
+ : core::ProcessorImpl(info),
s3_wrapper_(std::move(s3_request_sender)) {
}
void S3Processor::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) {
AwsProcessor::onSchedule(context, session_factory);
- if (!getProperty(Bucket.name)) {
+ if (!context.hasNonEmptyProperty(Bucket.name)) {
throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Bucket property missing or invalid");
}
}
diff --git a/extensions/aws/processors/S3Processor.h b/extensions/aws/processors/S3Processor.h
index 41beec73d..d3d0a485b 100644
--- a/extensions/aws/processors/S3Processor.h
+++ b/extensions/aws/processors/S3Processor.h
@@ -51,12 +51,13 @@ class S3Processor : public AwsProcessor {
EXTENSIONAPI static constexpr auto Properties = minifi::utils::array_cat(AwsProcessor::Properties, std::to_array<core::PropertyReference>({Bucket}));
+ using AwsProcessor::AwsProcessor;
- explicit S3Processor(std::string_view name, const minifi::utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger);
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
protected:
- explicit S3Processor(std::string_view name, const minifi::utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender);
+ explicit S3Processor(core::ProcessorMetadata info, std::unique_ptr<aws::s3::S3RequestSender> s3_request_sender);
+
aws::s3::S3Wrapper s3_wrapper_;
};
diff --git a/extensions/aws/tests/PutS3ObjectTests.cpp b/extensions/aws/tests/PutS3ObjectTests.cpp
index 9e5dbd807..a78b0cdd0 100644
--- a/extensions/aws/tests/PutS3ObjectTests.cpp
+++ b/extensions/aws/tests/PutS3ObjectTests.cpp
@@ -45,9 +45,7 @@ class PutS3ObjectLimitChanged : public minifi::aws::processors::PutS3Object {
protected:
friend class ::FlowProcessorS3TestsFixture<PutS3ObjectLimitChanged>;
- explicit PutS3ObjectLimitChanged(const std::string& name, const minifi::utils::Identifier& uuid, std::unique_ptr<minifi::aws::s3::S3RequestSender> s3_request_sender)
- : PutS3Object(name, uuid, std::move(s3_request_sender)) {
- }
+ using PutS3Object::PutS3Object;
uint64_t getMinPartSize() const override {
return 1;
diff --git a/extensions/aws/tests/S3TestsFixture.h b/extensions/aws/tests/S3TestsFixture.h
index 99493f3f9..f65030f84 100644
--- a/extensions/aws/tests/S3TestsFixture.h
+++ b/extensions/aws/tests/S3TestsFixture.h
@@ -135,7 +135,9 @@ class FlowProcessorS3TestsFixture : public S3TestsFixture<T> {
auto mock_s3_request_sender = std::make_unique<MockS3RequestSender>();
this->mock_s3_request_sender_ptr = mock_s3_request_sender.get();
- auto s3_processor_unique_ptr = std::unique_ptr<T>(new T("S3Processor", utils::Identifier(), std::move(mock_s3_request_sender)));
+ auto uuid = utils::IdGenerator::getIdGenerator()->generate();
+ auto impl = std::unique_ptr<T>(new T(core::ProcessorMetadata{.uuid = uuid, .name = "S3Processor", .logger = core::logging::LoggerFactory<T>::getLogger(uuid)}, std::move(mock_s3_request_sender)));
+ auto s3_processor_unique_ptr = std::make_unique<core::ProcessorProxy>("S3Processor", uuid, std::move(impl));
this->s3_processor = s3_processor_unique_ptr.get();
auto input_dir = this->test_controller.createTempDirectory();
@@ -197,7 +199,9 @@ class FlowProducerS3TestsFixture : public S3TestsFixture<T> {
FlowProducerS3TestsFixture() {
auto mock_s3_request_sender = std::make_unique<MockS3RequestSender>();
this->mock_s3_request_sender_ptr = mock_s3_request_sender.get();
- auto s3_processor_unique_ptr = std::unique_ptr<T>(new T("S3Processor", utils::Identifier(), std::move(mock_s3_request_sender)));
+ auto uuid = utils::IdGenerator::getIdGenerator()->generate();
+ auto impl = std::unique_ptr<T>(new T(core::ProcessorMetadata{.uuid = uuid, .name = "S3Processor", .logger = core::logging::LoggerFactory<T>::getLogger(uuid)}, std::move(mock_s3_request_sender)));
+ auto s3_processor_unique_ptr = std::make_unique<core::ProcessorProxy>("S3Processor", uuid, std::move(impl));
this->s3_processor = s3_processor_unique_ptr.get();
this->plan->addProcessor(
diff --git a/extensions/azure/processors/AzureBlobStorageProcessorBase.cpp b/extensions/azure/processors/AzureBlobStorageProcessorBase.cpp
index d3ec978e6..f0eee83a7 100644
--- a/extensions/azure/processors/AzureBlobStorageProcessorBase.cpp
+++ b/extensions/azure/processors/AzureBlobStorageProcessorBase.cpp
@@ -26,11 +26,11 @@
namespace org::apache::nifi::minifi::azure::processors {
void AzureBlobStorageProcessorBase::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) {
- if (!getProperty(ContainerName.name)) {
+ if (!context.hasNonEmptyProperty(ContainerName.name)) {
throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Container Name property missing or invalid");
}
- if (auto azure_storage_credentials_service = getProperty(AzureStorageCredentialsService.name)) {
+ if (auto azure_storage_credentials_service = context.getProperty(AzureStorageCredentialsService.name)) {
logger_->log_info("Getting Azure Storage credentials from controller service with name: '{}'", *azure_storage_credentials_service);
return;
}
@@ -42,21 +42,21 @@ void AzureBlobStorageProcessorBase::onSchedule(core::ProcessContext& context, co
return;
}
- if (auto connection_string = getProperty(ConnectionString.name); connection_string && !connection_string->empty()) {
+ if (context.hasNonEmptyProperty(ConnectionString.name)) {
logger_->log_info("Using connectionstring directly for Azure Storage authentication");
return;
}
- if (!getProperty(StorageAccountName.name)) {
+ if (!context.hasNonEmptyProperty(StorageAccountName.name)) {
throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Storage Account Name property missing or invalid");
}
- if (const auto storage_account_key = getProperty(StorageAccountKey.name); storage_account_key && !storage_account_key->empty()) {
+ if (context.hasNonEmptyProperty(StorageAccountKey.name)) {
logger_->log_info("Using storage account name and key for authentication");
return;
}
- if (const auto sas_token = getProperty(SASToken.name); !sas_token || sas_token->empty()) {
+ if (!context.hasNonEmptyProperty(SASToken.name)) {
throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Neither Storage Account Key nor SAS Token property was set.");
}
diff --git a/extensions/azure/processors/AzureBlobStorageProcessorBase.h b/extensions/azure/processors/AzureBlobStorageProcessorBase.h
index 6a63a20ab..b40db3aad 100644
--- a/extensions/azure/processors/AzureBlobStorageProcessorBase.h
+++ b/extensions/azure/processors/AzureBlobStorageProcessorBase.h
@@ -89,11 +89,9 @@ class AzureBlobStorageProcessorBase : public AzureStorageProcessorBase {
protected:
explicit AzureBlobStorageProcessorBase(
- std::string_view name,
- const minifi::utils::Identifier& uuid,
- const std::shared_ptr<core::logging::Logger>& logger,
+ core::ProcessorMetadata info,
std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureStorageProcessorBase(name, uuid, logger),
+ : AzureStorageProcessorBase(info),
azure_blob_storage_(std::move(blob_storage_client)) {
}
diff --git a/extensions/azure/processors/AzureBlobStorageSingleBlobProcessorBase.h b/extensions/azure/processors/AzureBlobStorageSingleBlobProcessorBase.h
index d65e16d06..bb1466249 100644
--- a/extensions/azure/processors/AzureBlobStorageSingleBlobProcessorBase.h
+++ b/extensions/azure/processors/AzureBlobStorageSingleBlobProcessorBase.h
@@ -41,11 +41,9 @@ class AzureBlobStorageSingleBlobProcessorBase : public AzureBlobStorageProcessor
protected:
explicit AzureBlobStorageSingleBlobProcessorBase(
- std::string_view name,
- const minifi::utils::Identifier& uuid,
- const std::shared_ptr<core::logging::Logger>& logger,
+ core::ProcessorMetadata info,
std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageProcessorBase(name, uuid, logger, std::move(blob_storage_client)) {
+ : AzureBlobStorageProcessorBase(info, std::move(blob_storage_client)) {
}
bool setBlobOperationParameters(
diff --git a/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h b/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
index 2b45b61bd..acb80568a 100644
--- a/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
+++ b/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
@@ -36,16 +36,13 @@ class AzureDataLakeStorageFileProcessorBase : public AzureDataLakeStorageProcess
.build();
EXTENSIONAPI static constexpr auto Properties = utils::array_cat(AzureDataLakeStorageProcessorBase::Properties, std::to_array<core::PropertyReference>({FileName}));
- explicit AzureDataLakeStorageFileProcessorBase(std::string_view name, const minifi::utils::Identifier& uuid, const std::shared_ptr<core::logging::Logger> &logger)
- : AzureDataLakeStorageProcessorBase(name, uuid, logger) {
- }
+ using AzureDataLakeStorageProcessorBase::AzureDataLakeStorageProcessorBase;
~AzureDataLakeStorageFileProcessorBase() override = default;
protected:
- explicit AzureDataLakeStorageFileProcessorBase(std::string_view name, const minifi::utils::Identifier& uuid, const std::shared_ptr<core::logging::Logger> &logger,
- std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageProcessorBase(name, uuid, logger, std::move(data_lake_storage_client)) {
+ explicit AzureDataLakeStorageFileProcessorBase(core::ProcessorMetadata info,std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageProcessorBase(info, std::move(data_lake_storage_client)) {
}
bool setFileOperationCommonParameters(
diff --git a/extensions/azure/processors/AzureDataLakeStorageProcessorBase.h b/extensions/azure/processors/AzureDataLakeStorageProcessorBase.h
index 3eb5aa125..35b4383dd 100644
--- a/extensions/azure/processors/AzureDataLakeStorageProcessorBase.h
+++ b/extensions/azure/processors/AzureDataLakeStorageProcessorBase.h
@@ -52,19 +52,15 @@ class AzureDataLakeStorageProcessorBase : public AzureStorageProcessorBase {
DirectoryName
}));
-
- explicit AzureDataLakeStorageProcessorBase(std::string_view name, const minifi::utils::Identifier& uuid, const std::shared_ptr<core::logging::Logger> &logger)
- : AzureStorageProcessorBase(name, uuid, logger) {
- }
+ using AzureStorageProcessorBase::AzureStorageProcessorBase;
~AzureDataLakeStorageProcessorBase() override = default;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& sessionFactory) override;
protected:
- explicit AzureDataLakeStorageProcessorBase(std::string_view name, const minifi::utils::Identifier& uuid, const std::shared_ptr<core::logging::Logger> &logger,
- std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureStorageProcessorBase(name, uuid, logger),
+ explicit AzureDataLakeStorageProcessorBase(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureStorageProcessorBase(info),
azure_data_lake_storage_(std::move(data_lake_storage_client)) {
}
diff --git a/extensions/azure/processors/AzureStorageProcessorBase.h b/extensions/azure/processors/AzureStorageProcessorBase.h
index c014895aa..13955ee4d 100644
--- a/extensions/azure/processors/AzureStorageProcessorBase.h
+++ b/extensions/azure/processors/AzureStorageProcessorBase.h
@@ -42,10 +42,7 @@ class AzureStorageProcessorBase : public core::ProcessorImpl {
.build();
EXTENSIONAPI static constexpr auto Properties = std::to_array<core::PropertyReference>({AzureStorageCredentialsService});
- AzureStorageProcessorBase(std::string_view name, const minifi::utils::Identifier& uuid, const std::shared_ptr<core::logging::Logger>& logger)
- : core::ProcessorImpl(name, uuid),
- logger_(logger) {
- }
+ using ProcessorImpl::ProcessorImpl;
protected:
enum class GetCredentialsFromControllerResult {
@@ -55,8 +52,6 @@ class AzureStorageProcessorBase : public core::ProcessorImpl {
};
std::tuple<GetCredentialsFromControllerResult, std::optional<storage::AzureStorageCredentials>> getCredentialsFromControllerService(core::ProcessContext &context) const;
-
- std::shared_ptr<core::logging::Logger> logger_;
};
} // namespace org::apache::nifi::minifi::azure::processors
diff --git a/extensions/azure/processors/DeleteAzureBlobStorage.h b/extensions/azure/processors/DeleteAzureBlobStorage.h
index 3bf2f3f81..bb5709179 100644
--- a/extensions/azure/processors/DeleteAzureBlobStorage.h
+++ b/extensions/azure/processors/DeleteAzureBlobStorage.h
@@ -62,8 +62,8 @@ class DeleteAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorB
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit DeleteAzureBlobStorage(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : DeleteAzureBlobStorage(name, uuid, nullptr) {
+ explicit DeleteAzureBlobStorage(core::ProcessorMetadata info)
+ : DeleteAzureBlobStorage(info, nullptr) {
}
void initialize() override;
@@ -73,8 +73,8 @@ class DeleteAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorB
private:
friend class ::AzureBlobStorageTestsFixture<DeleteAzureBlobStorage>;
- explicit DeleteAzureBlobStorage(std::string_view name, const minifi::utils::Identifier& uuid, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageSingleBlobProcessorBase(name, uuid, core::logging::LoggerFactory<DeleteAzureBlobStorage>::getLogger(), std::move(blob_storage_client)) {
+ explicit DeleteAzureBlobStorage(core::ProcessorMetadata info, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
+ : AzureBlobStorageSingleBlobProcessorBase(info, std::move(blob_storage_client)) {
}
std::optional<storage::DeleteAzureBlobStorageParameters> buildDeleteAzureBlobStorageParameters(
diff --git a/extensions/azure/processors/DeleteAzureDataLakeStorage.h b/extensions/azure/processors/DeleteAzureDataLakeStorage.h
index 770528a8a..88024909a 100644
--- a/extensions/azure/processors/DeleteAzureDataLakeStorage.h
+++ b/extensions/azure/processors/DeleteAzureDataLakeStorage.h
@@ -48,9 +48,7 @@ class DeleteAzureDataLakeStorage final : public AzureDataLakeStorageFileProcesso
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit DeleteAzureDataLakeStorage(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : AzureDataLakeStorageFileProcessorBase(name, uuid, core::logging::LoggerFactory<DeleteAzureDataLakeStorage>::getLogger(uuid)) {
- }
+ using AzureDataLakeStorageFileProcessorBase::AzureDataLakeStorageFileProcessorBase;
~DeleteAzureDataLakeStorage() override = default;
@@ -60,8 +58,8 @@ class DeleteAzureDataLakeStorage final : public AzureDataLakeStorageFileProcesso
private:
friend class ::AzureDataLakeStorageTestsFixture<DeleteAzureDataLakeStorage>;
- explicit DeleteAzureDataLakeStorage(std::string_view name, const minifi::utils::Identifier& uuid, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageFileProcessorBase(name, uuid, core::logging::LoggerFactory<DeleteAzureDataLakeStorage>::getLogger(), std::move(data_lake_storage_client)) {
+ explicit DeleteAzureDataLakeStorage(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageFileProcessorBase(info, std::move(data_lake_storage_client)) {
}
std::optional<storage::DeleteAzureDataLakeStorageParameters> buildDeleteParameters(core::ProcessContext& context, const core::FlowFile& flow_file);
diff --git a/extensions/azure/processors/FetchAzureBlobStorage.h b/extensions/azure/processors/FetchAzureBlobStorage.h
index 57111c54b..d64ec37dd 100644
--- a/extensions/azure/processors/FetchAzureBlobStorage.h
+++ b/extensions/azure/processors/FetchAzureBlobStorage.h
@@ -68,8 +68,8 @@ class FetchAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorBa
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit FetchAzureBlobStorage(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : FetchAzureBlobStorage(name, uuid, nullptr) {
+ explicit FetchAzureBlobStorage(core::ProcessorMetadata info)
+ : FetchAzureBlobStorage(info, nullptr) {
}
void initialize() override;
@@ -78,8 +78,8 @@ class FetchAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorBa
private:
friend class ::AzureBlobStorageTestsFixture<FetchAzureBlobStorage>;
- explicit FetchAzureBlobStorage(std::string_view name, const minifi::utils::Identifier& uuid, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageSingleBlobProcessorBase(name, uuid, core::logging::LoggerFactory<FetchAzureBlobStorage>::getLogger(), std::move(blob_storage_client)) {
+ explicit FetchAzureBlobStorage(core::ProcessorMetadata info, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
+ : AzureBlobStorageSingleBlobProcessorBase(info, std::move(blob_storage_client)) {
}
std::optional<storage::FetchAzureBlobStorageParameters> buildFetchAzureBlobStorageParameters(
diff --git a/extensions/azure/processors/FetchAzureDataLakeStorage.h b/extensions/azure/processors/FetchAzureDataLakeStorage.h
index 87d86473c..5626e8c28 100644
--- a/extensions/azure/processors/FetchAzureDataLakeStorage.h
+++ b/extensions/azure/processors/FetchAzureDataLakeStorage.h
@@ -69,9 +69,7 @@ class FetchAzureDataLakeStorage final : public AzureDataLakeStorageFileProcessor
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit FetchAzureDataLakeStorage(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : AzureDataLakeStorageFileProcessorBase(name, uuid, core::logging::LoggerFactory<FetchAzureDataLakeStorage>::getLogger(uuid)) {
- }
+ using AzureDataLakeStorageFileProcessorBase::AzureDataLakeStorageFileProcessorBase;
~FetchAzureDataLakeStorage() override = default;
@@ -81,8 +79,8 @@ class FetchAzureDataLakeStorage final : public AzureDataLakeStorageFileProcessor
private:
friend class ::AzureDataLakeStorageTestsFixture<FetchAzureDataLakeStorage>;
- explicit FetchAzureDataLakeStorage(std::string_view name, const minifi::utils::Identifier& uuid, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageFileProcessorBase(name, uuid, core::logging::LoggerFactory<FetchAzureDataLakeStorage>::getLogger(), std::move(data_lake_storage_client)) {
+ explicit FetchAzureDataLakeStorage(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageFileProcessorBase(info, std::move(data_lake_storage_client)) {
}
std::optional<storage::FetchAzureDataLakeStorageParameters> buildFetchParameters(core::ProcessContext& context, const core::FlowFile& flow_file);
diff --git a/extensions/azure/processors/ListAzureBlobStorage.h b/extensions/azure/processors/ListAzureBlobStorage.h
index a6ee36b39..96ce71c7c 100644
--- a/extensions/azure/processors/ListAzureBlobStorage.h
+++ b/extensions/azure/processors/ListAzureBlobStorage.h
@@ -64,12 +64,12 @@ class ListAzureBlobStorage final : public AzureBlobStorageProcessorBase {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit ListAzureBlobStorage(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : ListAzureBlobStorage(name, nullptr, uuid) {
+ explicit ListAzureBlobStorage(core::ProcessorMetadata info)
+ : ListAzureBlobStorage(info, nullptr) {
}
- explicit ListAzureBlobStorage(std::string_view name, std::unique_ptr<storage::BlobStorageClient> blob_storage_client, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : AzureBlobStorageProcessorBase(name, uuid, core::logging::LoggerFactory<ListAzureBlobStorage>::getLogger(uuid), std::move(blob_storage_client)) {
+ explicit ListAzureBlobStorage(core::ProcessorMetadata info, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
+ : AzureBlobStorageProcessorBase(info, std::move(blob_storage_client)) {
}
void initialize() override;
diff --git a/extensions/azure/processors/ListAzureDataLakeStorage.h b/extensions/azure/processors/ListAzureDataLakeStorage.h
index 513299b6e..fa45cf168 100644
--- a/extensions/azure/processors/ListAzureDataLakeStorage.h
+++ b/extensions/azure/processors/ListAzureDataLakeStorage.h
@@ -73,9 +73,7 @@ class ListAzureDataLakeStorage final : public AzureDataLakeStorageProcessorBase
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit ListAzureDataLakeStorage(std::string_view name, const minifi::utils::Identifier &uuid = minifi::utils::Identifier())
- : AzureDataLakeStorageProcessorBase(name, uuid, core::logging::LoggerFactory<ListAzureDataLakeStorage>::getLogger(uuid)) {
- }
+ using AzureDataLakeStorageProcessorBase::AzureDataLakeStorageProcessorBase;
~ListAzureDataLakeStorage() override = default;
@@ -86,8 +84,8 @@ class ListAzureDataLakeStorage final : public AzureDataLakeStorageProcessorBase
private:
friend class ::ListAzureDataLakeStorageTestsFixture;
- explicit ListAzureDataLakeStorage(std::string_view name, const minifi::utils::Identifier &uuid, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageProcessorBase(name, uuid, core::logging::LoggerFactory<ListAzureDataLakeStorage>::getLogger(uuid), std::move(data_lake_storage_client)) {
+ explicit ListAzureDataLakeStorage(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageProcessorBase(info, std::move(data_lake_storage_client)) {
}
std::optional<storage::ListAzureDataLakeStorageParameters> buildListParameters(core::ProcessContext &context);
diff --git a/extensions/azure/processors/PutAzureBlobStorage.h b/extensions/azure/processors/PutAzureBlobStorage.h
index 9a10c4ed2..695f4853a 100644
--- a/extensions/azure/processors/PutAzureBlobStorage.h
+++ b/extensions/azure/processors/PutAzureBlobStorage.h
@@ -65,8 +65,8 @@ class PutAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorBase
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PutAzureBlobStorage(std::string_view name, const minifi::utils::Identifier& uuid = minifi::utils::Identifier())
- : PutAzureBlobStorage(name, uuid, nullptr) {
+ explicit PutAzureBlobStorage(core::ProcessorMetadata info)
+ : PutAzureBlobStorage(info, nullptr) {
}
void initialize() override;
@@ -107,8 +107,8 @@ class PutAzureBlobStorage final : public AzureBlobStorageSingleBlobProcessorBase
private:
friend class ::AzureBlobStorageTestsFixture<PutAzureBlobStorage>;
- explicit PutAzureBlobStorage(std::string_view name, const minifi::utils::Identifier& uuid, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
- : AzureBlobStorageSingleBlobProcessorBase(name, uuid, core::logging::LoggerFactory<PutAzureBlobStorage>::getLogger(), std::move(blob_storage_client)) {
+ explicit PutAzureBlobStorage(core::ProcessorMetadata info, std::unique_ptr<storage::BlobStorageClient> blob_storage_client)
+ : AzureBlobStorageSingleBlobProcessorBase(info, std::move(blob_storage_client)) {
}
std::optional<storage::PutAzureBlobStorageParameters> buildPutAzureBlobStorageParameters(core::ProcessContext &context, const core::FlowFile& flow_file);
diff --git a/extensions/azure/processors/PutAzureDataLakeStorage.h b/extensions/azure/processors/PutAzureDataLakeStorage.h
index a89aaeaf8..f1684c003 100644
--- a/extensions/azure/processors/PutAzureDataLakeStorage.h
+++ b/extensions/azure/processors/PutAzureDataLakeStorage.h
@@ -69,9 +69,7 @@ class PutAzureDataLakeStorage final : public AzureDataLakeStorageFileProcessorBa
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PutAzureDataLakeStorage(std::string_view name, const minifi::utils::Identifier &uuid = minifi::utils::Identifier())
- : PutAzureDataLakeStorage(name, uuid, nullptr) {
- }
+ using AzureDataLakeStorageFileProcessorBase::AzureDataLakeStorageFileProcessorBase;
void initialize() override;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
@@ -97,8 +95,8 @@ class PutAzureDataLakeStorage final : public AzureDataLakeStorageFileProcessorBa
std::shared_ptr<core::logging::Logger> logger_;
};
- explicit PutAzureDataLakeStorage(std::string_view name, const minifi::utils::Identifier &uuid, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
- : AzureDataLakeStorageFileProcessorBase(name, uuid, core::logging::LoggerFactory<PutAzureDataLakeStorage>::getLogger(), std::move(data_lake_storage_client)) {
+ explicit PutAzureDataLakeStorage(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ : AzureDataLakeStorageFileProcessorBase(info, std::move(data_lake_storage_client)) {
}
std::optional<storage::PutAzureDataLakeStorageParameters> buildUploadParameters(core::ProcessContext &context, const core::FlowFile& flow_file);
diff --git a/extensions/azure/tests/AzureBlobStorageTestsFixture.h b/extensions/azure/tests/AzureBlobStorageTestsFixture.h
index 7ca59170d..68d278711 100644
--- a/extensions/azure/tests/AzureBlobStorageTestsFixture.h
+++ b/extensions/azure/tests/AzureBlobStorageTestsFixture.h
@@ -33,6 +33,7 @@
#include "processors/UpdateAttribute.h"
#include "utils/file/FileUtils.h"
#include "MockBlobStorage.h"
+#include "core/ProcessorProxy.h"
const std::string CONTAINER_NAME = "test-container";
const std::string STORAGE_ACCOUNT_NAME = "test-account";
@@ -61,8 +62,10 @@ class AzureBlobStorageTestsFixture {
plan_ = test_controller_.createPlan();
auto mock_blob_storage = std::make_unique<MockBlobStorage>();
mock_blob_storage_ptr_ = mock_blob_storage.get();
- auto azure_blob_storage_processor_unique_ptr = std::unique_ptr<ProcessorType>(
- new ProcessorType("AzureBlobStorageProcessor", utils::Identifier(), std::move(mock_blob_storage)));
+ auto uuid = utils::IdGenerator::getIdGenerator()->generate();
+ auto impl = std::unique_ptr<ProcessorType>(
+ new ProcessorType({.uuid = uuid, .name = "AzureBlobStorageProcessor", .logger = logging::LoggerFactory<ProcessorType>::getLogger(uuid)}, std::move(mock_blob_storage)));
+ auto azure_blob_storage_processor_unique_ptr = std::make_unique<core::ProcessorProxy>(impl->getName(), impl->getUUID(), std::move(impl));
azure_blob_storage_processor_ = azure_blob_storage_processor_unique_ptr.get();
auto input_dir = test_controller_.createTempDirectory();
std::ofstream input_file_stream(input_dir / GET_FILE_NAME);
diff --git a/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h b/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
index 4206ef52f..e49b3959a 100644
--- a/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
+++ b/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
@@ -60,8 +60,10 @@ class AzureDataLakeStorageTestsFixture {
plan_ = test_controller_.createPlan();
auto mock_data_lake_storage_client = std::make_unique<MockDataLakeStorageClient>();
mock_data_lake_storage_client_ptr_ = mock_data_lake_storage_client.get();
- auto azure_data_lake_storage_unique_ptr = std::unique_ptr<AzureDataLakeStorageProcessor>(
- new AzureDataLakeStorageProcessor("AzureDataLakeStorageProcessor", utils::Identifier(), std::move(mock_data_lake_storage_client)));
+ auto uuid = utils::IdGenerator::getIdGenerator()->generate();
+ auto impl = std::unique_ptr<AzureDataLakeStorageProcessor>(
+ new AzureDataLakeStorageProcessor({.uuid = uuid, .name = "AzureDataLakeStorageProcessor", .logger = logging::LoggerFactory<AzureDataLakeStorageProcessor>::getLogger(uuid)}, std::move(mock_data_lake_storage_client)));
+ auto azure_data_lake_storage_unique_ptr = std::make_unique<core::ProcessorProxy>(impl->getName(), impl->getUUID(), std::move(impl));
azure_data_lake_storage_ = azure_data_lake_storage_unique_ptr.get();
auto input_dir = test_controller_.createTempDirectory();
minifi::test::utils::putFileToDir(input_dir, GETFILE_FILE_NAME, TEST_DATA);
diff --git a/extensions/azure/tests/ListAzureBlobStorageTests.cpp b/extensions/azure/tests/ListAzureBlobStorageTests.cpp
index 3977df86d..a63517035 100644
--- a/extensions/azure/tests/ListAzureBlobStorageTests.cpp
+++ b/extensions/azure/tests/ListAzureBlobStorageTests.cpp
@@ -45,7 +45,9 @@ class ListAzureBlobStorageTestsFixture {
plan_ = test_controller_.createPlan();
auto mock_blob_storage = std::make_unique<MockBlobStorage>();
mock_blob_storage_ptr_ = mock_blob_storage.get();
- auto list_azure_blob_storage_unique_ptr = std::make_unique<minifi::azure::processors::ListAzureBlobStorage>("ListAzureBlobStorage", std::move(mock_blob_storage));
+ auto uuid = utils::IdGenerator::getIdGenerator()->generate();
+ auto impl = std::make_unique<minifi::azure::processors::ListAzureBlobStorage>(core::ProcessorMetadata{.uuid = uuid, .name = "ListAzureBlobStorage", .logger = logging::LoggerFactory<minifi::azure::processors::ListAzureBlobStorage>::getLogger(uuid)}, std::move(mock_blob_storage));
+ auto list_azure_blob_storage_unique_ptr = std::make_unique<core::ProcessorProxy>(impl->getName(), impl->getUUID(), std::move(impl));
list_azure_blob_storage_ = list_azure_blob_storage_unique_ptr.get();
plan_->addProcessor(std::move(list_azure_blob_storage_unique_ptr), "ListAzureBlobStorage", { {"success", "d"} });
diff --git a/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp b/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
index ebd89aa49..ba89a8b5b 100644
--- a/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
+++ b/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
@@ -42,8 +42,10 @@ class ListAzureDataLakeStorageTestsFixture {
plan_ = test_controller_.createPlan();
auto mock_data_lake_storage_client = std::make_unique<MockDataLakeStorageClient>();
mock_data_lake_storage_client_ptr_ = mock_data_lake_storage_client.get();
- auto list_azure_data_lake_storage_unique_ptr = std::unique_ptr<minifi::azure::processors::ListAzureDataLakeStorage>(
- new minifi::azure::processors::ListAzureDataLakeStorage("ListAzureDataLakeStorage", utils::Identifier(), std::move(mock_data_lake_storage_client)));
+ auto uuid = utils::IdGenerator::getIdGenerator()->generate();
+ auto impl = std::unique_ptr<minifi::azure::processors::ListAzureDataLakeStorage>(
+ new minifi::azure::processors::ListAzureDataLakeStorage({.uuid = uuid, .name = "ListAzureDataLakeStorage", .logger = logging::LoggerFactory<minifi::azure::processors::ListAzureDataLakeStorage>::getLogger(uuid)}, std::move(mock_data_lake_storage_client)));
+ auto list_azure_data_lake_storage_unique_ptr = std::make_unique<core::ProcessorProxy>(impl->getName(), impl->getUUID(), std::move(impl));
list_azure_data_lake_storage_ = list_azure_data_lake_storage_unique_ptr.get();
plan_->addProcessor(std::move(list_azure_data_lake_storage_unique_ptr), "ListAzureDataLakeStorage", { {"success", "d"} });
diff --git a/extensions/bustache/ApplyTemplate.h b/extensions/bustache/ApplyTemplate.h
index f1390856a..dfdcfece0 100644
--- a/extensions/bustache/ApplyTemplate.h
+++ b/extensions/bustache/ApplyTemplate.h
@@ -23,7 +23,7 @@
#include <string>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -34,10 +34,7 @@ namespace org::apache::nifi::minifi::processors {
*/
class ApplyTemplate final : public core::ProcessorImpl {
public:
- explicit ApplyTemplate(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ApplyTemplate>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Applies the mustache template specified by the \"Template\" property and writes the output to the flow file content. "
"FlowFile attributes are used as template parameters.";
diff --git a/extensions/bustache/tests/ApplyTemplateTests.cpp b/extensions/bustache/tests/ApplyTemplateTests.cpp
index 1af8bcb32..0f697d741 100644
--- a/extensions/bustache/tests/ApplyTemplateTests.cpp
+++ b/extensions/bustache/tests/ApplyTemplateTests.cpp
@@ -22,6 +22,7 @@
#include <iostream>
#include "unit/TestBase.h"
+#include "unit/TestUtils.h"
#include "unit/Catch.h"
#include "core/Core.h"
@@ -47,7 +48,7 @@ const char* EXPECT_OUTPUT = "TemplateBegins\nExampleValue\nTemplateEnds";
TEST_CASE("Test Creation of ApplyTemplate", "[ApplyTemplateCreate]") {
TestController testController;
- std::shared_ptr<core::Processor> processor = std::make_shared<org::apache::nifi::minifi::processors::ApplyTemplate>("processor_name");
+ std::shared_ptr<core::Processor> processor = minifi::test::utils::make_processor<org::apache::nifi::minifi::processors::ApplyTemplate>("processor_name");
REQUIRE(processor->getName() == "processor_name");
REQUIRE(processor->getUUID());
}
diff --git a/extensions/civetweb/processors/ListenHTTP.cpp b/extensions/civetweb/processors/ListenHTTP.cpp
index 46f627f34..e3d7f18b1 100644
--- a/extensions/civetweb/processors/ListenHTTP.cpp
+++ b/extensions/civetweb/processors/ListenHTTP.cpp
@@ -57,7 +57,7 @@ void ListenHTTP::onSchedule(core::ProcessContext& context, core::ProcessSessionF
std::string headersAsAttributesPattern = context.getProperty(HeadersAsAttributesRegex).value_or("");
logger_->log_debug("ListenHTTP using {}: {}", HeadersAsAttributesRegex.name, headersAsAttributesPattern);
- auto numThreads = getMaxConcurrentTasks();
+ auto numThreads = context.getMaxConcurrentTasks();
logger_->log_info("ListenHTTP starting HTTP server on port {} and path {} with {} threads", randomPort ? "random" : listeningPort, base_path, numThreads);
@@ -134,7 +134,7 @@ void ListenHTTP::onSchedule(core::ProcessContext& context, core::ProcessSessionF
ListenHTTP::~ListenHTTP() = default;
-void ListenHTTP::onTrigger(core::ProcessContext&, core::ProcessSession& session) {
+void ListenHTTP::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
logger_->log_trace("OnTrigger ListenHTTP");
bool restored_processed = false;
for (auto& ff : file_store_.getNewFlowFiles()) {
@@ -146,7 +146,7 @@ void ListenHTTP::onTrigger(core::ProcessContext&, core::ProcessSession& session)
const bool incoming_processed = processIncomingFlowFile(session);
const bool request_processed = processRequestBuffer(session);
if (!restored_processed && !incoming_processed && !request_processed) {
- yield();
+ context.yield();
}
}
diff --git a/extensions/civetweb/processors/ListenHTTP.h b/extensions/civetweb/processors/ListenHTTP.h
index 4a5181988..199b2155b 100644
--- a/extensions/civetweb/processors/ListenHTTP.h
+++ b/extensions/civetweb/processors/ListenHTTP.h
@@ -27,7 +27,6 @@
#include <CivetServer.h>
#include "FlowFileRecord.h"
-#include "core/Processor.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -40,6 +39,7 @@
#include "utils/Export.h"
#include "utils/RegexUtils.h"
#include "core/FlowFileStore.h"
+#include "core/ProcessorImpl.h"
namespace org::apache::nifi::minifi::test {
struct ListenHTTPTestAccessor;
@@ -55,9 +55,8 @@ class ListenHTTP : public core::ProcessorImpl {
public:
friend struct ::org::apache::nifi::minifi::test::ListenHTTPTestAccessor;
- explicit ListenHTTP(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ListenHTTP>::getLogger(uuid_);
+ explicit ListenHTTP(core::ProcessorMetadata info)
+ : ProcessorImpl(info) {
callbacks_.log_message = &logMessage;
callbacks_.log_access = &logAccess;
}
diff --git a/extensions/civetweb/tests/C2VerifyServeResults.cpp b/extensions/civetweb/tests/C2VerifyServeResults.cpp
index bd15c9d3b..3a355eee2 100644
--- a/extensions/civetweb/tests/C2VerifyServeResults.cpp
+++ b/extensions/civetweb/tests/C2VerifyServeResults.cpp
@@ -62,10 +62,8 @@ class VerifyC2Server : public HTTPIntegrationBase {
auto proc = pg->findProcessorByName("invoke");
REQUIRE(proc != nullptr);
- auto inv = dynamic_cast<minifi::processors::InvokeHTTP*>(proc);
-
- REQUIRE(inv != nullptr);
- std::string url = inv->getProperty(processors::InvokeHTTP::URL.name).value_or("");
+ proc->getImpl<minifi::processors::InvokeHTTP>();
+ std::string url = proc->getProperty(processors::InvokeHTTP::URL.name).value_or("");
std::string port;
std::string scheme;
diff --git a/extensions/civetweb/tests/ListenHTTPTests.cpp b/extensions/civetweb/tests/ListenHTTPTests.cpp
index b439886ec..0e0b439d2 100644
--- a/extensions/civetweb/tests/ListenHTTPTests.cpp
+++ b/extensions/civetweb/tests/ListenHTTPTests.cpp
@@ -147,9 +147,8 @@ class ListenHTTPTestsFixture {
plan->runNextProcessor(); // UpdateAttribute
plan->runNextProcessor(); // ListenHTTP
- auto raw_ptr = dynamic_cast<org::apache::nifi::minifi::processors::ListenHTTP*>(listen_http);
- std::string protocol = std::string("http") + (raw_ptr->isSecure() ? "s" : "");
- std::string portstr = raw_ptr->getPort();
+ std::string protocol = std::string("http") + (listen_http.get().isSecure() ? "s" : "");
+ std::string portstr = listen_http.get().getPort();
REQUIRE(LogTestController::getInstance().contains("Listening on port " + portstr));
url = protocol + "://localhost:" + portstr + "/contentListener/" + endpoint;
@@ -203,13 +202,13 @@ class ListenHTTPTestsFixture {
if (client_to_use) {
REQUIRE(response_expectations.size() == 1);
}
- auto* proc = dynamic_cast<minifi::processors::ListenHTTP*>(&plan->getCurrentContext()->getProcessor());
+ TypedProcessorWrapper<minifi::processors::ListenHTTP> proc = &plan->getCurrentContext()->getProcessor();
REQUIRE(proc);
std::vector<std::thread> client_threads;
for (auto& expect : response_expectations) {
- size_t prev_req_count = ListenHTTPTestAccessor::call_pendingRequestCount(*proc);
+ size_t prev_req_count = ListenHTTPTestAccessor::call_pendingRequestCount(proc.get());
auto thread_done_flag = std::make_shared<std::atomic_bool>(false);
client_threads.emplace_back([&, thread_done_flag] {
auto client = client_to_use ? std::move(client_to_use) : initialize_client();
@@ -217,7 +216,7 @@ class ListenHTTPTestsFixture {
check_response(client->submit(), expect, *client);
thread_done_flag->store(true);
});
- while (!thread_done_flag->load() && ListenHTTPTestAccessor::call_pendingRequestCount(*proc) != prev_req_count + 1) {
+ while (!thread_done_flag->load() && ListenHTTPTestAccessor::call_pendingRequestCount(proc.get()) != prev_req_count + 1) {
std::this_thread::sleep_for(1ms);
}
}
@@ -256,7 +255,7 @@ class ListenHTTPTestsFixture {
std::shared_ptr<TestPlan> plan;
core::Processor* get_file = nullptr;
core::Processor* update_attribute = nullptr;
- core::Processor* listen_http = nullptr;
+ TypedProcessorWrapper<org::apache::nifi::minifi::processors::ListenHTTP> listen_http = nullptr;
core::Processor* log_attribute = nullptr;
std::shared_ptr<minifi::controllers::SSLContextServiceInterface> ssl_context_service;
@@ -272,7 +271,7 @@ class ListenHTTPTestsFixture {
TEST_CASE("ListenHTTP creation", "[basic]") {
TestController testController;
std::shared_ptr<core::Processor>
- processor = std::make_shared<org::apache::nifi::minifi::processors::ListenHTTP>("processorname");
+ processor = minifi::test::utils::make_processor<org::apache::nifi::minifi::processors::ListenHTTP>("processorname");
REQUIRE(processor->getName() == "processorname");
}
@@ -697,7 +696,7 @@ TEST_CASE_METHOD(ListenHTTPTestsFixture, "HTTPS minimum SSL version", "[https]")
TEST_CASE("ListenHTTP bored yield", "[listenhttp][bored][yield]") {
using processors::ListenHTTP;
- SingleProcessorTestController controller{std::make_unique<ListenHTTP>("listenhttp")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenHTTP>("listenhttp")};
auto listen_http = controller.getProcessor();
REQUIRE(listen_http->setProperty(ListenHTTP::Port.name, "0"));
diff --git a/extensions/couchbase/tests/GetCouchbaseKeyTests.cpp b/extensions/couchbase/tests/GetCouchbaseKeyTests.cpp
index 0a7080bcb..aa3e3f442 100644
--- a/extensions/couchbase/tests/GetCouchbaseKeyTests.cpp
+++ b/extensions/couchbase/tests/GetCouchbaseKeyTests.cpp
@@ -17,6 +17,7 @@
*/
#include "unit/TestBase.h"
+#include "unit/TestUtils.h"
#include "unit/Catch.h"
#include "unit/SingleProcessorTestController.h"
#include "processors/GetCouchbaseKey.h"
@@ -38,7 +39,7 @@ struct ExpectedCallOptions {
class GetCouchbaseKeyTestController : public TestController {
public:
GetCouchbaseKeyTestController()
- : controller_(std::make_unique<processors::GetCouchbaseKey>("GetCouchbaseKey")),
+ : controller_(minifi::test::utils::make_processor<processors::GetCouchbaseKey>("GetCouchbaseKey")),
proc_(controller_.getProcessor()) {
LogTestController::getInstance().setDebug<TestPlan>();
LogTestController::getInstance().setDebug<minifi::core::Processor>();
diff --git a/extensions/couchbase/tests/PutCouchbaseKeyTests.cpp b/extensions/couchbase/tests/PutCouchbaseKeyTests.cpp
index 492f4bab9..51d180053 100644
--- a/extensions/couchbase/tests/PutCouchbaseKeyTests.cpp
+++ b/extensions/couchbase/tests/PutCouchbaseKeyTests.cpp
@@ -17,6 +17,7 @@
*/
#include "unit/TestBase.h"
+#include "unit/TestUtils.h"
#include "unit/Catch.h"
#include "unit/SingleProcessorTestController.h"
#include "processors/PutCouchbaseKey.h"
@@ -42,7 +43,7 @@ struct ExpectedCallOptions {
class PutCouchbaseKeyTestController : public TestController {
public:
PutCouchbaseKeyTestController()
- : controller_(std::make_unique<processors::PutCouchbaseKey>("PutCouchbaseKey")),
+ : controller_(minifi::test::utils::make_processor<processors::PutCouchbaseKey>("PutCouchbaseKey")),
proc_(controller_.getProcessor()) {
LogTestController::getInstance().setDebug<TestPlan>();
LogTestController::getInstance().setDebug<minifi::core::Processor>();
diff --git a/extensions/elasticsearch/PostElasticsearch.h b/extensions/elasticsearch/PostElasticsearch.h
index 6f92d8799..9d3cb4dc8 100644
--- a/extensions/elasticsearch/PostElasticsearch.h
+++ b/extensions/elasticsearch/PostElasticsearch.h
@@ -24,7 +24,7 @@
#include "controllers/SSLContextServiceInterface.h"
#include "ElasticsearchCredentialsControllerService.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -38,10 +38,7 @@ class PostElasticsearch : public core::ProcessorImpl {
public:
EXTENSIONAPI static constexpr const char* Description = "An Elasticsearch/Opensearch post processor that uses the Elasticsearch/Opensearch _bulk REST API.";
- explicit PostElasticsearch(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<PostElasticsearch>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~PostElasticsearch() override = default;
EXTENSIONAPI static constexpr auto Action = core::PropertyDefinitionBuilder<>::createProperty("Action")
diff --git a/extensions/elasticsearch/tests/PostElasticsearchTests.cpp b/extensions/elasticsearch/tests/PostElasticsearchTests.cpp
index 6eb68522f..83d1f6555 100644
--- a/extensions/elasticsearch/tests/PostElasticsearchTests.cpp
+++ b/extensions/elasticsearch/tests/PostElasticsearchTests.cpp
@@ -21,12 +21,13 @@
#include "unit/SingleProcessorTestController.h"
#include "unit/Catch.h"
#include "core/FlowFile.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::extensions::elasticsearch::test {
TEST_CASE("PostElasticsearch", "[elastic]") {
MockElastic mock_elastic("10433");
- minifi::test::SingleProcessorTestController test_controller{std::make_unique<PostElasticsearch>("PostElasticsearch")};
+ minifi::test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<PostElasticsearch>("PostElasticsearch")};
auto post_elasticsearch_json = test_controller.getProcessor();
auto elasticsearch_credentials_controller_service = test_controller.plan->addController("ElasticsearchCredentialsControllerService", "elasticsearch_credentials_controller_service");
CHECK(test_controller.plan->setProperty(post_elasticsearch_json,
diff --git a/extensions/execute-process/ExecuteProcess.cpp b/extensions/execute-process/ExecuteProcess.cpp
index a4c2188d2..a569ee083 100644
--- a/extensions/execute-process/ExecuteProcess.cpp
+++ b/extensions/execute-process/ExecuteProcess.cpp
@@ -79,11 +79,11 @@ std::vector<std::string> ExecuteProcess::readArgs() const {
return args;
}
-void ExecuteProcess::executeProcessForkFailed() {
+void ExecuteProcess::executeProcessForkFailed(core::ProcessContext& context) {
logger_->log_error("Execute Process fork failed");
close(pipefd_[0]);
close(pipefd_[1]);
- yield();
+ context.yield();
}
void ExecuteProcess::executeChildProcess() const {
@@ -208,26 +208,26 @@ void ExecuteProcess::collectChildProcessOutput(core::ProcessSession& session) {
pid_ = 0;
}
-void ExecuteProcess::onTrigger(core::ProcessContext&, core::ProcessSession& session) {
+void ExecuteProcess::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
if (full_command_.empty()) {
- yield();
+ context.yield();
return;
}
std::error_code current_path_error;
std::filesystem::current_path(working_dir_, current_path_error);
if (current_path_error) {
- yield();
+ context.yield();
return;
}
logger_->log_info("Execute Command {}", full_command_);
if (pipe(pipefd_) == -1) {
- yield();
+ context.yield();
return;
}
switch (pid_ = fork()) {
case -1:
- executeProcessForkFailed();
+ executeProcessForkFailed(context);
break;
case 0: // this is the code the child runs
executeChildProcess();
diff --git a/extensions/execute-process/ExecuteProcess.h b/extensions/execute-process/ExecuteProcess.h
index 0deba66c9..c8b186afd 100644
--- a/extensions/execute-process/ExecuteProcess.h
+++ b/extensions/execute-process/ExecuteProcess.h
@@ -36,7 +36,7 @@
#include <utility>
#include "core/Core.h"
#include "core/logging/LoggerFactory.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -48,11 +48,7 @@ namespace org::apache::nifi::minifi::processors {
class ExecuteProcess final : public core::ProcessorImpl {
public:
- explicit ExecuteProcess(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid),
- working_dir_(".") {
- logger_ = core::logging::LoggerFactory<ExecuteProcess>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~ExecuteProcess() override {
if (pid_ > 0) {
kill(pid_, SIGTERM);
@@ -107,7 +103,7 @@ class ExecuteProcess final : public core::ProcessorImpl {
private:
std::vector<std::string> readArgs() const;
- void executeProcessForkFailed();
+ void executeProcessForkFailed(core::ProcessContext& context);
void executeChildProcess() const;
void collectChildProcessOutput(core::ProcessSession& session);
void readOutputInBatches(core::ProcessSession& session) const;
@@ -116,7 +112,7 @@ class ExecuteProcess final : public core::ProcessorImpl {
std::string command_;
std::string command_argument_;
- std::filesystem::path working_dir_;
+ std::filesystem::path working_dir_{"."};
std::chrono::milliseconds batch_duration_ = std::chrono::milliseconds(0);
bool redirect_error_stream_ = false;
std::string full_command_;
diff --git a/extensions/execute-process/test/ExecuteProcessTests.cpp b/extensions/execute-process/test/ExecuteProcessTests.cpp
index d0838c6ea..9dbe86b45 100644
--- a/extensions/execute-process/test/ExecuteProcessTests.cpp
+++ b/extensions/execute-process/test/ExecuteProcessTests.cpp
@@ -20,6 +20,7 @@
#include "../ExecuteProcess.h"
#include "unit/SingleProcessorTestController.h"
#include "utils/file/FileUtils.h"
+#include "unit/TestUtils.h"
using namespace std::literals::chrono_literals;
@@ -28,13 +29,13 @@ namespace org::apache::nifi::minifi::test {
class ExecuteProcessTestsFixture {
public:
ExecuteProcessTestsFixture()
- : controller_(std::make_unique<processors::ExecuteProcess>("ExecuteProcess")),
- execute_process_(controller_.getProcessor<processors::ExecuteProcess>()) {
+ : controller_(minifi::test::utils::make_processor<processors::ExecuteProcess>("ExecuteProcess")),
+ execute_process_(controller_.getProcessor()) {
LogTestController::getInstance().setTrace<processors::ExecuteProcess>();
}
protected:
test::SingleProcessorTestController controller_;
- processors::ExecuteProcess* execute_process_;
+ core::Processor* execute_process_;
};
TEST_CASE_METHOD(ExecuteProcessTestsFixture, "ExecuteProcess can run a single command", "[ExecuteProcess]") {
diff --git a/extensions/expression-language/ProcessContextExpr.cpp b/extensions/expression-language/ProcessContextExpr.cpp
index bff17c6b6..04deec7f6 100644
--- a/extensions/expression-language/ProcessContextExpr.cpp
+++ b/extensions/expression-language/ProcessContextExpr.cpp
@@ -25,47 +25,42 @@
namespace org::apache::nifi::minifi::core {
-nonstd::expected<std::string, std::error_code> ProcessContextExpr::getProperty(ProcessContext& context, const std::string_view name, const FlowFile* flow_file) const {
- const auto property = context.getProcessorInfo().getPropertyReference(name);
+nonstd::expected<std::string, std::error_code> ProcessContextExpr::getProperty(const std::string_view name, const FlowFile* flow_file) const {
+ const auto property = getProcessorInfo().getPropertyReference(name);
if (!property) {
return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty);
}
if (!property->supports_expression_language) {
- return context.getProperty(name, flow_file);
+ return ProcessContextImpl::getProperty(name, flow_file);
}
if (!cached_expressions_.contains(name)) {
- auto expression_str = context.getProperty(name, flow_file);
+ auto expression_str = ProcessContextImpl::getProperty(name, flow_file);
if (!expression_str) { return expression_str; }
cached_expressions_.emplace(std::string{name}, expression::compile(*expression_str));
}
- expression::Parameters p(&context, flow_file);
+ expression::Parameters p(this, flow_file);
auto result = cached_expressions_[std::string{name}](p).asString();
- if (!property->getValidator().validate(result)) {
+ if (!property->validator->validate(result)) {
return nonstd::make_unexpected(PropertyErrorCode::ValidationFailed);
}
return result;
}
-nonstd::expected<std::string, std::error_code> ProcessContextExpr::getDynamicProperty(ProcessContext& context, const std::string_view name, const FlowFile* flow_file) const {
+nonstd::expected<std::string, std::error_code> ProcessContextExpr::getDynamicProperty(const std::string_view name, const FlowFile* flow_file) const {
if (!cached_dynamic_expressions_.contains(name)) {
- auto expression_str = context.getDynamicProperty(name, flow_file);
+ auto expression_str = ProcessContextImpl::getDynamicProperty(name, flow_file);
if (!expression_str) { return expression_str; }
cached_dynamic_expressions_.emplace(std::string{name}, expression::compile(*expression_str));
}
- const expression::Parameters p(&context, flow_file);
+ const expression::Parameters p(this, flow_file);
return cached_dynamic_expressions_[std::string{name}](p).asString();
}
-//nonstd::expected<void, std::error_code> ProcessContextExpr::setProperty(const std::string_view name, std::string value) {
-// cached_expressions_.erase(std::string{name});
-// return ProcessContextImpl::setProperty(name, std::move(value));
-//}
-//
-//nonstd::expected<void, std::error_code> ProcessContextExpr::setDynamicProperty(std::string name, std::string value) {
-// cached_dynamic_expressions_.erase(name);
-// return ProcessContextImpl::setDynamicProperty(std::move(name), std::move(value));
-//}
+nonstd::expected<void, std::error_code> ProcessContextExpr::setProperty(const std::string_view name, std::string value) {
+ cached_expressions_.erase(std::string{name});
+ return ProcessContextImpl::setProperty(name, std::move(value));
+}
nonstd::expected<void, std::error_code> ProcessContextExpr::setDynamicProperty(std::string name, std::string value) {
cached_dynamic_expressions_.erase(name);
diff --git a/extensions/expression-language/ProcessContextExpr.h b/extensions/expression-language/ProcessContextExpr.h
index cb33c973c..228b1e35f 100644
--- a/extensions/expression-language/ProcessContextExpr.h
+++ b/extensions/expression-language/ProcessContextExpr.h
@@ -34,15 +34,16 @@ namespace org::apache::nifi::minifi::core {
*/
class ProcessContextExpr final : public core::ProcessContextImpl {
public:
+ using ProcessContextImpl::ProcessContextImpl;
~ProcessContextExpr() override = default;
- nonstd::expected<std::string, std::error_code> getProperty(ProcessContext& context, std::string_view name, const FlowFile*) const override;
- nonstd::expected<std::string, std::error_code> getDynamicProperty(ProcessContext& context, std::string_view name, const FlowFile*) const override;
+ nonstd::expected<std::string, std::error_code> getProperty(std::string_view name, const FlowFile*) const override;
+ nonstd::expected<std::string, std::error_code> getDynamicProperty(std::string_view name, const FlowFile*) const override;
//
-// nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value);
-// nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value);
+ nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value) override;
+ nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value) override;
- std::map<std::string, std::string> getDynamicProperties(ProcessContext& context, const FlowFile*) const override;
+ std::map<std::string, std::string> getDynamicProperties(const FlowFile*) const override;
private:
mutable std::unordered_map<std::string, expression::Expression, utils::string::transparent_string_hash, std::equal_to<>> cached_expressions_;
diff --git a/extensions/expression-language/tests/ProcessContextExprTests.cpp b/extensions/expression-language/tests/ProcessContextExprTests.cpp
index 2d9531ff1..595a79033 100644
--- a/extensions/expression-language/tests/ProcessContextExprTests.cpp
+++ b/extensions/expression-language/tests/ProcessContextExprTests.cpp
@@ -21,7 +21,7 @@
#include "core/ProcessContext.h"
#include "core/Resource.h"
#include "ProcessContextExpr.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/RelationshipDefinition.h"
diff --git a/extensions/gcp/processors/DeleteGCSObject.h b/extensions/gcp/processors/DeleteGCSObject.h
index 93e71c766..0bffde16b 100644
--- a/extensions/gcp/processors/DeleteGCSObject.h
+++ b/extensions/gcp/processors/DeleteGCSObject.h
@@ -33,9 +33,7 @@ namespace org::apache::nifi::minifi::extensions::gcp {
class DeleteGCSObject : public GCSProcessor {
public:
- explicit DeleteGCSObject(std::string_view name, const utils::Identifier& uuid = {})
- : GCSProcessor(name, uuid, core::logging::LoggerFactory<DeleteGCSObject>::getLogger(uuid)) {
- }
+ using GCSProcessor::GCSProcessor;
~DeleteGCSObject() override = default;
EXTENSIONAPI static constexpr const char* Description = "Deletes an object from a Google Cloud Bucket.";
diff --git a/extensions/gcp/processors/FetchGCSObject.h b/extensions/gcp/processors/FetchGCSObject.h
index cd030b237..f645506b1 100644
--- a/extensions/gcp/processors/FetchGCSObject.h
+++ b/extensions/gcp/processors/FetchGCSObject.h
@@ -33,9 +33,7 @@ namespace org::apache::nifi::minifi::extensions::gcp {
class FetchGCSObject : public GCSProcessor {
public:
- explicit FetchGCSObject(std::string_view name, const utils::Identifier& uuid = {})
- : GCSProcessor(name, uuid, core::logging::LoggerFactory<FetchGCSObject>::getLogger(uuid)) {
- }
+ using GCSProcessor::GCSProcessor;
~FetchGCSObject() override = default;
EXTENSIONAPI static constexpr const char* Description = "Fetches a file from a Google Cloud Bucket. Designed to be used in tandem with ListGCSBucket.";
diff --git a/extensions/gcp/processors/GCSProcessor.h b/extensions/gcp/processors/GCSProcessor.h
index b15327329..430c5f04a 100644
--- a/extensions/gcp/processors/GCSProcessor.h
+++ b/extensions/gcp/processors/GCSProcessor.h
@@ -23,7 +23,7 @@
#include "../controllerservices/GCPCredentialsControllerService.h"
#include "core/logging/Logger.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -34,10 +34,7 @@
namespace org::apache::nifi::minifi::extensions::gcp {
class GCSProcessor : public core::ProcessorImpl {
public:
- GCSProcessor(std::string_view name, const minifi::utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger)
- : core::ProcessorImpl(name, uuid),
- logger_(std::move(logger)) {
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr auto GCPCredentials = core::PropertyDefinitionBuilder<>::createProperty("GCP Credentials Provider Service")
.withDescription("The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService.")
@@ -72,7 +69,6 @@ class GCSProcessor : public core::ProcessorImpl {
std::optional<std::string> endpoint_url_;
std::shared_ptr<google::cloud::storage::oauth2::Credentials> gcp_credentials_;
google::cloud::storage::RetryPolicyOption::Type retry_policy_ = std::make_shared<google::cloud::storage::LimitedErrorCountRetryPolicy>(6);
- std::shared_ptr<core::logging::Logger> logger_;
};
} // namespace org::apache::nifi::minifi::extensions::gcp
diff --git a/extensions/gcp/processors/ListGCSBucket.h b/extensions/gcp/processors/ListGCSBucket.h
index 8aa6b26f5..e384808d1 100644
--- a/extensions/gcp/processors/ListGCSBucket.h
+++ b/extensions/gcp/processors/ListGCSBucket.h
@@ -43,9 +43,7 @@ inline constexpr auto FILENAME_OUTPUT_ATTRIBUTE_DESCRIPTION = utils::array_to_st
class ListGCSBucket : public GCSProcessor {
public:
- explicit ListGCSBucket(std::string_view name, const utils::Identifier& uuid = {})
- : GCSProcessor(name, uuid, core::logging::LoggerFactory<ListGCSBucket>::getLogger(uuid)) {
- }
+ using GCSProcessor::GCSProcessor;
~ListGCSBucket() override = default;
EXTENSIONAPI static constexpr const char* Description = "Retrieves a listing of objects from an GCS bucket. "
diff --git a/extensions/gcp/processors/PutGCSObject.h b/extensions/gcp/processors/PutGCSObject.h
index 8d46736b1..3a0fe3792 100644
--- a/extensions/gcp/processors/PutGCSObject.h
+++ b/extensions/gcp/processors/PutGCSObject.h
@@ -72,9 +72,7 @@ namespace org::apache::nifi::minifi::extensions::gcp {
class PutGCSObject : public GCSProcessor {
public:
- explicit PutGCSObject(std::string_view name, const utils::Identifier& uuid = {})
- : GCSProcessor(name, uuid, core::logging::LoggerFactory<PutGCSObject>::getLogger(uuid)) {
- }
+ using GCSProcessor::GCSProcessor;
~PutGCSObject() override = default;
EXTENSIONAPI static constexpr const char* Description = "Puts flow files to a Google Cloud Storage Bucket.";
diff --git a/extensions/gcp/tests/DeleteGCSObjectTests.cpp b/extensions/gcp/tests/DeleteGCSObjectTests.cpp
index 465cd6baa..f4498d730 100644
--- a/extensions/gcp/tests/DeleteGCSObjectTests.cpp
+++ b/extensions/gcp/tests/DeleteGCSObjectTests.cpp
@@ -22,6 +22,7 @@
#include "google/cloud/storage/testing/mock_client.h"
#include "google/cloud/storage/internal/object_metadata_parser.h"
#include "google/cloud/storage/testing/canonical_errors.h"
+#include "unit/TestUtils.h"
namespace gcs = ::google::cloud::storage;
namespace minifi_gcp = org::apache::nifi::minifi::extensions::gcp;
@@ -49,7 +50,7 @@ REGISTER_RESOURCE(DeleteGCSObjectMocked, Processor);
class DeleteGCSObjectTests : public ::testing::Test {
public:
void SetUp() override {
- delete_gcs_object_ = test_controller_.getProcessor<DeleteGCSObjectMocked>();
+ delete_gcs_object_ = test_controller_.getProcessor();
gcp_credentials_node_ = test_controller_.plan->addController("GCPCredentialsControllerService", "gcp_credentials_controller_service");
test_controller_.plan->setProperty(gcp_credentials_node_,
GCPCredentialsControllerService::CredentialsLoc,
@@ -59,13 +60,13 @@ class DeleteGCSObjectTests : public ::testing::Test {
"gcp_credentials_controller_service");
}
- org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{std::make_unique<DeleteGCSObjectMocked>("DeleteGCSObjectMocked")};
+ org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{minifi::test::utils::make_processor<DeleteGCSObjectMocked>("DeleteGCSObjectMocked")};
std::shared_ptr<minifi::core::controller::ControllerServiceNode> gcp_credentials_node_;
- DeleteGCSObjectMocked* delete_gcs_object_ = nullptr;
+ TypedProcessorWrapper<DeleteGCSObjectMocked> delete_gcs_object_ = nullptr;
};
TEST_F(DeleteGCSObjectTests, MissingBucket) {
- EXPECT_CALL(*delete_gcs_object_->mock_client_, CreateResumableUpload).Times(0);
+ EXPECT_CALL(*delete_gcs_object_.get().mock_client_, CreateResumableUpload).Times(0);
EXPECT_TRUE(test_controller_.plan->setProperty(delete_gcs_object_, DeleteGCSObject::Bucket, ""));
const auto& result = test_controller_.trigger("hello world");
EXPECT_EQ(0, result.at(DeleteGCSObject::Success).size());
@@ -76,7 +77,7 @@ TEST_F(DeleteGCSObjectTests, MissingBucket) {
}
TEST_F(DeleteGCSObjectTests, ServerGivesPermaError) {
- EXPECT_CALL(*delete_gcs_object_->mock_client_, DeleteObject)
+ EXPECT_CALL(*delete_gcs_object_.get().mock_client_, DeleteObject)
.WillOnce(testing::Return(PermanentError()));
EXPECT_TRUE(test_controller_.plan->setProperty(delete_gcs_object_, DeleteGCSObject::Bucket, "bucket-from-property"));
const auto& result = test_controller_.trigger("hello world");
@@ -88,7 +89,7 @@ TEST_F(DeleteGCSObjectTests, ServerGivesPermaError) {
}
TEST_F(DeleteGCSObjectTests, ServerGivesTransientErrors) {
- EXPECT_CALL(*delete_gcs_object_->mock_client_, DeleteObject).WillOnce(testing::Return(TransientError()));
+ EXPECT_CALL(*delete_gcs_object_.get().mock_client_, DeleteObject).WillOnce(testing::Return(TransientError()));
EXPECT_TRUE(test_controller_.plan->setProperty(delete_gcs_object_, DeleteGCSObject::NumberOfRetries, "1"));
EXPECT_TRUE(test_controller_.plan->setProperty(delete_gcs_object_, DeleteGCSObject::Bucket, "bucket-from-property"));
const auto& result = test_controller_.trigger("hello world", {{std::string(minifi_gcp::GCS_BUCKET_ATTR), "bucket-from-attribute"}});
@@ -101,7 +102,7 @@ TEST_F(DeleteGCSObjectTests, ServerGivesTransientErrors) {
TEST_F(DeleteGCSObjectTests, HandlingSuccessfullDeletion) {
- EXPECT_CALL(*delete_gcs_object_->mock_client_, DeleteObject)
+ EXPECT_CALL(*delete_gcs_object_.get().mock_client_, DeleteObject)
.WillOnce([](DeleteObjectRequest const& request) {
EXPECT_EQ("bucket-from-attribute", request.bucket_name());
EXPECT_TRUE(request.HasOption<gcs::Generation>());
@@ -117,7 +118,7 @@ TEST_F(DeleteGCSObjectTests, HandlingSuccessfullDeletion) {
}
TEST_F(DeleteGCSObjectTests, EmptyGeneration) {
- EXPECT_CALL(*delete_gcs_object_->mock_client_, DeleteObject)
+ EXPECT_CALL(*delete_gcs_object_.get().mock_client_, DeleteObject)
.WillOnce([](DeleteObjectRequest const& request) {
EXPECT_EQ("bucket-from-attribute", request.bucket_name());
EXPECT_FALSE(request.HasOption<gcs::Generation>());
diff --git a/extensions/gcp/tests/FetchGCSObjectTests.cpp b/extensions/gcp/tests/FetchGCSObjectTests.cpp
index 4323528fb..c233f8d67 100644
--- a/extensions/gcp/tests/FetchGCSObjectTests.cpp
+++ b/extensions/gcp/tests/FetchGCSObjectTests.cpp
@@ -22,6 +22,7 @@
#include "google/cloud/storage/testing/mock_client.h"
#include "google/cloud/storage/internal/object_metadata_parser.h"
#include "google/cloud/storage/testing/canonical_errors.h"
+#include "unit/TestUtils.h"
namespace gcs = ::google::cloud::storage;
namespace minifi_gcp = org::apache::nifi::minifi::extensions::gcp;
@@ -46,7 +47,7 @@ REGISTER_RESOURCE(FetchGCSObjectMocked, Processor);
class FetchGCSObjectTests : public ::testing::Test {
public:
void SetUp() override {
- fetch_gcs_object_ = test_controller_.getProcessor<FetchGCSObjectMocked>();
+ fetch_gcs_object_ = test_controller_.getProcessor();
gcp_credentials_node_ = test_controller_.plan->addController("GCPCredentialsControllerService", "gcp_credentials_controller_service");
test_controller_.plan->setProperty(gcp_credentials_node_,
GCPCredentialsControllerService::CredentialsLoc,
@@ -55,13 +56,13 @@ class FetchGCSObjectTests : public ::testing::Test {
FetchGCSObject::GCPCredentials,
"gcp_credentials_controller_service");
}
- FetchGCSObjectMocked* fetch_gcs_object_ = nullptr;
- org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{std::make_unique<FetchGCSObjectMocked>("FetchGCSObjectMocked")};
+ TypedProcessorWrapper<FetchGCSObjectMocked> fetch_gcs_object_;
+ org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{minifi::test::utils::make_processor<FetchGCSObjectMocked>("FetchGCSObjectMocked")};
std::shared_ptr<minifi::core::controller::ControllerServiceNode> gcp_credentials_node_;
};
TEST_F(FetchGCSObjectTests, MissingBucket) {
- EXPECT_CALL(*fetch_gcs_object_->mock_client_, CreateResumableUpload).Times(0);
+ EXPECT_CALL(*fetch_gcs_object_.get().mock_client_, CreateResumableUpload).Times(0);
EXPECT_TRUE(test_controller_.plan->setProperty(fetch_gcs_object_, FetchGCSObject::Bucket, ""));
const auto& result = test_controller_.trigger("hello world");
EXPECT_EQ(0, result.at(FetchGCSObject::Success).size());
@@ -72,7 +73,7 @@ TEST_F(FetchGCSObjectTests, MissingBucket) {
}
TEST_F(FetchGCSObjectTests, ServerError) {
- EXPECT_CALL(*fetch_gcs_object_->mock_client_, ReadObject)
+ EXPECT_CALL(*fetch_gcs_object_.get().mock_client_, ReadObject)
.WillOnce([](gcs::internal::ReadObjectRangeRequest const& request) {
EXPECT_EQ(request.bucket_name(), "bucket-from-property") << request;
auto mock_source = std::make_unique<gcs::testing::MockObjectReadSource>();
@@ -106,7 +107,7 @@ TEST_F(FetchGCSObjectTests, HappyPath) {
return gcs::internal::ReadSourceResult{
l, gcs::internal::HttpResponse{200, {}, {}}};
};
- EXPECT_CALL(*fetch_gcs_object_->mock_client_, ReadObject)
+ EXPECT_CALL(*fetch_gcs_object_.get().mock_client_, ReadObject)
.WillOnce([&](gcs::internal::ReadObjectRangeRequest const& request) {
EXPECT_EQ(request.bucket_name(), "bucket-from-attribute") << request;
EXPECT_TRUE(request.HasOption<gcs::Generation>());
@@ -140,7 +141,7 @@ TEST_F(FetchGCSObjectTests, EmptyGeneration) {
return gcs::internal::ReadSourceResult{
l, gcs::internal::HttpResponse{200, {}, {}}};
};
- EXPECT_CALL(*fetch_gcs_object_->mock_client_, ReadObject)
+ EXPECT_CALL(*fetch_gcs_object_.get().mock_client_, ReadObject)
.WillOnce([&](gcs::internal::ReadObjectRangeRequest const& request) {
EXPECT_EQ(request.bucket_name(), "bucket-from-attribute") << request;
EXPECT_FALSE(request.HasOption<gcs::Generation>());
diff --git a/extensions/gcp/tests/ListGCSBucketTests.cpp b/extensions/gcp/tests/ListGCSBucketTests.cpp
index f0fbc3d57..e0b435a13 100644
--- a/extensions/gcp/tests/ListGCSBucketTests.cpp
+++ b/extensions/gcp/tests/ListGCSBucketTests.cpp
@@ -21,6 +21,7 @@
#include "google/cloud/storage/testing/mock_client.h"
#include "google/cloud/storage/internal/object_metadata_parser.h"
#include "google/cloud/storage/testing/canonical_errors.h"
+#include "unit/TestUtils.h"
namespace gcs = ::google::cloud::storage;
namespace minifi_gcp = org::apache::nifi::minifi::extensions::gcp;
@@ -65,7 +66,7 @@ auto CreateObject(int index, int generation = 1) {
class ListGCSBucketTests : public ::testing::Test {
public:
void SetUp() override {
- list_gcs_bucket_ = test_controller_.getProcessor<ListGCSBucketMocked>();
+ list_gcs_bucket_ = test_controller_.getProcessor();
gcp_credentials_node_ = test_controller_.plan->addController("GCPCredentialsControllerService", "gcp_credentials_controller_service");
test_controller_.plan->setProperty(gcp_credentials_node_,
GCPCredentialsControllerService::CredentialsLoc,
@@ -74,13 +75,13 @@ class ListGCSBucketTests : public ::testing::Test {
ListGCSBucket::GCPCredentials,
"gcp_credentials_controller_service");
}
- org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{std::make_unique<ListGCSBucketMocked>("ListGCSBucketMocked")};
- ListGCSBucketMocked* list_gcs_bucket_ = nullptr;
+ org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{minifi::test::utils::make_processor<ListGCSBucketMocked>("ListGCSBucketMocked")};
+ TypedProcessorWrapper<ListGCSBucketMocked> list_gcs_bucket_;
std::shared_ptr<minifi::core::controller::ControllerServiceNode> gcp_credentials_node_;
};
TEST_F(ListGCSBucketTests, MissingBucket) {
- EXPECT_CALL(*list_gcs_bucket_->mock_client_, CreateResumableUpload).Times(0);
+ EXPECT_CALL(*list_gcs_bucket_.get().mock_client_, CreateResumableUpload).Times(0);
EXPECT_THROW(test_controller_.trigger(), std::runtime_error);
}
@@ -88,7 +89,7 @@ TEST_F(ListGCSBucketTests, ServerGivesPermaError) {
auto return_permanent_error = [](ListObjectsRequest const&) {
return google::cloud::StatusOr<ListObjectsResponse>(PermanentError());
};
- EXPECT_CALL(*list_gcs_bucket_->mock_client_, ListObjects)
+ EXPECT_CALL(*list_gcs_bucket_.get().mock_client_, ListObjects)
.WillOnce(return_permanent_error);
EXPECT_TRUE(test_controller_.plan->setProperty(list_gcs_bucket_, ListGCSBucket::Bucket, "bucket-from-property"));
const auto& result = test_controller_.trigger();
@@ -99,7 +100,7 @@ TEST_F(ListGCSBucketTests, ServerGivesTransientErrors) {
auto return_temp_error = [](ListObjectsRequest const&) {
return google::cloud::StatusOr<ListObjectsResponse>(TransientError());
};
- EXPECT_CALL(*list_gcs_bucket_->mock_client_, ListObjects).WillOnce(return_temp_error);
+ EXPECT_CALL(*list_gcs_bucket_.get().mock_client_, ListObjects).WillOnce(return_temp_error);
EXPECT_TRUE(test_controller_.plan->setProperty(list_gcs_bucket_, ListGCSBucket::NumberOfRetries, "1"));
EXPECT_TRUE(test_controller_.plan->setProperty(list_gcs_bucket_, ListGCSBucket::Bucket, "bucket-from-property"));
const auto& result = test_controller_.trigger();
@@ -107,7 +108,7 @@ TEST_F(ListGCSBucketTests, ServerGivesTransientErrors) {
}
TEST_F(ListGCSBucketTests, WithoutVersions) {
- EXPECT_CALL(*list_gcs_bucket_->mock_client_, ListObjects)
+ EXPECT_CALL(*list_gcs_bucket_.get().mock_client_, ListObjects)
.WillOnce([](ListObjectsRequest const& req)
-> google::cloud::StatusOr<ListObjectsResponse> {
EXPECT_EQ("bucket-from-property", req.bucket_name());
@@ -127,7 +128,7 @@ TEST_F(ListGCSBucketTests, WithoutVersions) {
TEST_F(ListGCSBucketTests, WithVersions) {
- EXPECT_CALL(*list_gcs_bucket_->mock_client_, ListObjects)
+ EXPECT_CALL(*list_gcs_bucket_.get().mock_client_, ListObjects)
.WillOnce([](ListObjectsRequest const& req)
-> google::cloud::StatusOr<ListObjectsResponse> {
EXPECT_EQ("bucket-from-property", req.bucket_name());
diff --git a/extensions/gcp/tests/PutGCSObjectTests.cpp b/extensions/gcp/tests/PutGCSObjectTests.cpp
index 80970b78a..f099b3d77 100644
--- a/extensions/gcp/tests/PutGCSObjectTests.cpp
+++ b/extensions/gcp/tests/PutGCSObjectTests.cpp
@@ -24,6 +24,7 @@
#include "google/cloud/storage/internal/object_metadata_parser.h"
#include "google/cloud/storage/retry_policy.h"
#include "google/cloud/storage/testing/canonical_errors.h"
+#include "unit/TestUtils.h"
namespace gcs = ::google::cloud::storage;
namespace minifi_gcp = org::apache::nifi::minifi::extensions::gcp;
@@ -52,7 +53,7 @@ REGISTER_RESOURCE(PutGCSObjectMocked, Processor);
class PutGCSObjectTests : public ::testing::Test {
public:
void SetUp() override {
- put_gcs_object_ = test_controller_.getProcessor<PutGCSObjectMocked>();
+ put_gcs_object_ = test_controller_.getProcessor();
gcp_credentials_node_ = test_controller_.plan->addController("GCPCredentialsControllerService", "gcp_credentials_controller_service");
test_controller_.plan->setProperty(gcp_credentials_node_,
GCPCredentialsControllerService::CredentialsLoc,
@@ -61,8 +62,8 @@ class PutGCSObjectTests : public ::testing::Test {
PutGCSObject::GCPCredentials,
"gcp_credentials_controller_service");
}
- PutGCSObjectMocked* put_gcs_object_ = nullptr;
- org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{std::make_unique<PutGCSObjectMocked>("PutGCSObjectMocked")};
+ TypedProcessorWrapper<PutGCSObjectMocked> put_gcs_object_;
+ org::apache::nifi::minifi::test::SingleProcessorTestController test_controller_{minifi::test::utils::make_processor<PutGCSObjectMocked>("PutGCSObjectMocked")};
std::shared_ptr<minifi::core::controller::ControllerServiceNode> gcp_credentials_node_;
static auto return_upload_done(const ResumableUploadRequest& request) {
@@ -80,7 +81,7 @@ class PutGCSObjectTests : public ::testing::Test {
};
TEST_F(PutGCSObjectTests, MissingBucket) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload).Times(0);
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload).Times(0);
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, ""));
const auto& result = test_controller_.trigger("hello world");
EXPECT_EQ(0, result.at(PutGCSObject::Success).size());
@@ -91,10 +92,10 @@ TEST_F(PutGCSObjectTests, MissingBucket) {
}
TEST_F(PutGCSObjectTests, BucketFromAttribute) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload)
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload)
.WillOnce([this](const ResumableUploadRequest& request) {
EXPECT_EQ("bucket-from-attribute", request.bucket_name());
- EXPECT_CALL(*put_gcs_object_->mock_client_, UploadChunk).WillOnce(return_upload_done(request));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, UploadChunk).WillOnce(return_upload_done(request));
return gcs::internal::CreateResumableUploadResponse{"test-only-upload-id"};
});
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, "${gcs.bucket}"));
@@ -105,7 +106,7 @@ TEST_F(PutGCSObjectTests, BucketFromAttribute) {
}
TEST_F(PutGCSObjectTests, ServerGivesTransientErrors) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload).WillOnce(testing::Return(TransientError()));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload).WillOnce(testing::Return(TransientError()));
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::NumberOfRetries, "2"));
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, "bucket-from-property"));
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Key, "object-name-from-property"));
@@ -118,7 +119,7 @@ TEST_F(PutGCSObjectTests, ServerGivesTransientErrors) {
}
TEST_F(PutGCSObjectTests, ServerGivesPermaError) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload).WillOnce(testing::Return(PermanentError()));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload).WillOnce(testing::Return(PermanentError()));
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, "bucket-from-property"));
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Key, "object-name-from-property"));
const auto& result = test_controller_.trigger("hello world");
@@ -130,13 +131,13 @@ TEST_F(PutGCSObjectTests, ServerGivesPermaError) {
}
TEST_F(PutGCSObjectTests, NonRequiredPropertiesAreMissing) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload)
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload)
.WillOnce([this](const ResumableUploadRequest& request) {
EXPECT_FALSE(request.HasOption<gcs::MD5HashValue>());
EXPECT_FALSE(request.HasOption<gcs::Crc32cChecksumValue>());
EXPECT_FALSE(request.HasOption<gcs::PredefinedAcl>());
EXPECT_FALSE(request.HasOption<gcs::IfGenerationMatch>());
- EXPECT_CALL(*put_gcs_object_->mock_client_, UploadChunk).WillOnce(return_upload_done(request));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, UploadChunk).WillOnce(return_upload_done(request));
return gcs::internal::CreateResumableUploadResponse{"test-only-upload-id"};
});
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, "bucket-from-property"));
@@ -147,13 +148,13 @@ TEST_F(PutGCSObjectTests, NonRequiredPropertiesAreMissing) {
}
TEST_F(PutGCSObjectTests, Crc32cMD5LocationTest) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload)
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload)
.WillOnce([this](const ResumableUploadRequest& request) {
EXPECT_TRUE(request.HasOption<gcs::Crc32cChecksumValue>());
EXPECT_EQ("yZRlqg==", request.GetOption<gcs::Crc32cChecksumValue>().value());
EXPECT_TRUE(request.HasOption<gcs::MD5HashValue>());
EXPECT_EQ("XrY7u+Ae7tCTyyK7j1rNww==", request.GetOption<gcs::MD5HashValue>().value());
- EXPECT_CALL(*put_gcs_object_->mock_client_, UploadChunk).WillOnce(return_upload_done(request));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, UploadChunk).WillOnce(return_upload_done(request));
return gcs::internal::CreateResumableUploadResponse{"test-only-upload-id"};
});
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::MD5Hash, "${md5}"));
@@ -166,10 +167,10 @@ TEST_F(PutGCSObjectTests, Crc32cMD5LocationTest) {
}
TEST_F(PutGCSObjectTests, DontOverwriteTest) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload)
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload)
.WillOnce([this](const ResumableUploadRequest& request) {
EXPECT_TRUE(request.HasOption<gcs::IfGenerationMatch>());
- EXPECT_CALL(*put_gcs_object_->mock_client_, UploadChunk).WillOnce(return_upload_done(request));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, UploadChunk).WillOnce(return_upload_done(request));
return gcs::internal::CreateResumableUploadResponse{"test-only-upload-id"};
});
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::OverwriteObject, "false"));
@@ -182,10 +183,10 @@ TEST_F(PutGCSObjectTests, DontOverwriteTest) {
}
TEST_F(PutGCSObjectTests, ValidServerSideEncryptionTest) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload)
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload)
.WillOnce([this](const ResumableUploadRequest& request) {
EXPECT_TRUE(request.HasOption<gcs::EncryptionKey>());
- EXPECT_CALL(*put_gcs_object_->mock_client_, UploadChunk).WillOnce(return_upload_done(request));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, UploadChunk).WillOnce(return_upload_done(request));
return gcs::internal::CreateResumableUploadResponse{"test-only-upload-id"};
});
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::EncryptionKey, "ZW5jcnlwdGlvbl9rZXk="));
@@ -200,7 +201,7 @@ TEST_F(PutGCSObjectTests, ValidServerSideEncryptionTest) {
}
TEST_F(PutGCSObjectTests, InvalidServerSideEncryptionTest) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload).Times(0);
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload).Times(0);
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::EncryptionKey, "not_base64_key"));
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, "bucket-from-property"));
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Key, "object-name-from-property"));
@@ -208,10 +209,10 @@ TEST_F(PutGCSObjectTests, InvalidServerSideEncryptionTest) {
}
TEST_F(PutGCSObjectTests, NoContentType) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload)
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload)
.WillOnce([this](const ResumableUploadRequest& request) {
EXPECT_FALSE(request.HasOption<gcs::ContentType>());
- EXPECT_CALL(*put_gcs_object_->mock_client_, UploadChunk).WillOnce(return_upload_done(request));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, UploadChunk).WillOnce(return_upload_done(request));
return gcs::internal::CreateResumableUploadResponse{"test-only-upload-id"};
});
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, "bucket-from-property"));
@@ -223,11 +224,11 @@ TEST_F(PutGCSObjectTests, NoContentType) {
}
TEST_F(PutGCSObjectTests, ContentTypeFromAttribute) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload)
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload)
.WillOnce([this](const ResumableUploadRequest& request) {
EXPECT_TRUE(request.HasOption<gcs::ContentType>());
EXPECT_EQ("text/attribute", request.GetOption<gcs::ContentType>().value());
- EXPECT_CALL(*put_gcs_object_->mock_client_, UploadChunk).WillOnce(return_upload_done(request));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, UploadChunk).WillOnce(return_upload_done(request));
return gcs::internal::CreateResumableUploadResponse{"test-only-upload-id"};
});
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, "bucket-from-property"));
@@ -239,11 +240,11 @@ TEST_F(PutGCSObjectTests, ContentTypeFromAttribute) {
}
TEST_F(PutGCSObjectTests, ObjectACLTest) {
- EXPECT_CALL(*put_gcs_object_->mock_client_, CreateResumableUpload)
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, CreateResumableUpload)
.WillOnce([this](const ResumableUploadRequest& request) {
EXPECT_TRUE(request.HasOption<gcs::PredefinedAcl>());
EXPECT_EQ(gcs::PredefinedAcl::AuthenticatedRead().value(), request.GetOption<gcs::PredefinedAcl>().value());
- EXPECT_CALL(*put_gcs_object_->mock_client_, UploadChunk).WillOnce(return_upload_done(request));
+ EXPECT_CALL(*put_gcs_object_.get().mock_client_, UploadChunk).WillOnce(return_upload_done(request));
return gcs::internal::CreateResumableUploadResponse{"test-only-upload-id"};
});
EXPECT_TRUE(test_controller_.plan->setProperty(put_gcs_object_, PutGCSObject::Bucket, "bucket-from-property"));
diff --git a/extensions/grafana-loki/PushGrafanaLoki.h b/extensions/grafana-loki/PushGrafanaLoki.h
index 13e8f577f..7ca192aa1 100644
--- a/extensions/grafana-loki/PushGrafanaLoki.h
+++ b/extensions/grafana-loki/PushGrafanaLoki.h
@@ -22,7 +22,7 @@
#include <map>
#include "controllers/SSLContextServiceInterface.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -34,10 +34,9 @@ namespace org::apache::nifi::minifi::extensions::grafana::loki {
class PushGrafanaLoki : public core::ProcessorImpl {
public:
- PushGrafanaLoki(const std::string_view name, const utils::Identifier& uuid, const std::shared_ptr<core::logging::Logger>& logger)
- : ProcessorImpl(name, uuid),
- logger_(logger),
- log_batch_(logger_) {
+ PushGrafanaLoki(core::ProcessorMetadata info)
+ : ProcessorImpl(info),
+ log_batch_(info.logger) {
}
~PushGrafanaLoki() override = default;
@@ -141,7 +140,6 @@ class PushGrafanaLoki : public core::ProcessorImpl {
void setUpStateManager(core::ProcessContext& context);
virtual void setUpStreamLabels(core::ProcessContext& context) = 0;
- std::shared_ptr<core::logging::Logger> logger_;
std::optional<uint64_t> max_batch_size_;
std::vector<std::string> log_line_metadata_attributes_;
bool log_line_batch_size_is_set_ = false;
diff --git a/extensions/grafana-loki/PushGrafanaLokiGrpc.h b/extensions/grafana-loki/PushGrafanaLokiGrpc.h
index 146c0d46c..808f5b77d 100644
--- a/extensions/grafana-loki/PushGrafanaLokiGrpc.h
+++ b/extensions/grafana-loki/PushGrafanaLokiGrpc.h
@@ -31,9 +31,7 @@ class PushGrafanaLokiGrpc final : public PushGrafanaLoki {
EXTENSIONAPI static constexpr const char* Description = "A Grafana Loki push processor that uses the Grafana Loki Grpc port. The processor expects each flow file to contain a single log line to be "
"pushed to Grafana Loki, therefore it is usually used together with the TailFile processor.";
- explicit PushGrafanaLokiGrpc(const std::string_view name, const utils::Identifier& uuid = {})
- : PushGrafanaLoki(name, uuid, core::logging::LoggerFactory<PushGrafanaLokiGrpc>::getLogger(uuid)) {
- }
+ using PushGrafanaLoki::PushGrafanaLoki;
~PushGrafanaLokiGrpc() override = default;
EXTENSIONAPI static constexpr auto KeepAliveTime = core::PropertyDefinitionBuilder<>::createProperty("Keep Alive Time")
diff --git a/extensions/grafana-loki/PushGrafanaLokiREST.h b/extensions/grafana-loki/PushGrafanaLokiREST.h
index ddd5374f8..93448b09c 100644
--- a/extensions/grafana-loki/PushGrafanaLokiREST.h
+++ b/extensions/grafana-loki/PushGrafanaLokiREST.h
@@ -28,9 +28,7 @@ class PushGrafanaLokiREST : public PushGrafanaLoki {
EXTENSIONAPI static constexpr const char* Description = "A Grafana Loki push processor that uses the Grafana Loki REST API. The processor expects each flow file to contain a single log line to be "
"pushed to Grafana Loki, therefore it is usually used together with the TailFile processor.";
- explicit PushGrafanaLokiREST(const std::string& name, const utils::Identifier& uuid = {})
- : PushGrafanaLoki(name, uuid, core::logging::LoggerFactory<PushGrafanaLokiREST>::getLogger(uuid)) {
- }
+ using PushGrafanaLoki::PushGrafanaLoki;
~PushGrafanaLokiREST() override = default;
EXTENSIONAPI static constexpr auto ReadTimeout = core::PropertyDefinitionBuilder<>::createProperty("Read Timeout")
diff --git a/extensions/grafana-loki/tests/PushGrafanaLokiGrpcTest.cpp b/extensions/grafana-loki/tests/PushGrafanaLokiGrpcTest.cpp
index 8b6c5e4ae..d583cf60c 100644
--- a/extensions/grafana-loki/tests/PushGrafanaLokiGrpcTest.cpp
+++ b/extensions/grafana-loki/tests/PushGrafanaLokiGrpcTest.cpp
@@ -25,7 +25,7 @@
namespace org::apache::nifi::minifi::extensions::grafana::loki::test {
TEST_CASE("Url property is required", "[PushGrafanaLokiGrpc]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<PushGrafanaLokiGrpc>("PushGrafanaLokiGrpc"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<PushGrafanaLokiGrpc>("PushGrafanaLokiGrpc"));
auto push_grafana_loki_grpc = test_controller.getProcessor();
test_controller.plan->setProperty(push_grafana_loki_grpc, PushGrafanaLokiGrpc::Url, "");
test_controller.plan->setProperty(push_grafana_loki_grpc, PushGrafanaLokiGrpc::StreamLabels, "job=minifi,directory=/opt/minifi/logs/");
@@ -34,7 +34,7 @@ TEST_CASE("Url property is required", "[PushGrafanaLokiGrpc]") {
}
TEST_CASE("Valid stream labels need to be set", "[PushGrafanaLokiGrpc]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<PushGrafanaLokiGrpc>("PushGrafanaLokiGrpc"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<PushGrafanaLokiGrpc>("PushGrafanaLokiGrpc"));
auto push_grafana_loki_grpc = test_controller.getProcessor();
test_controller.plan->setProperty(push_grafana_loki_grpc, PushGrafanaLokiGrpc::Url, "localhost:10991");
test_controller.plan->setProperty(push_grafana_loki_grpc, PushGrafanaLokiGrpc::LogLineBatchSize, "1");
@@ -48,7 +48,7 @@ TEST_CASE("Valid stream labels need to be set", "[PushGrafanaLokiGrpc]") {
}
TEST_CASE("Log Line Batch Size cannot be 0", "[PushGrafanaLokiGrpc]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<PushGrafanaLokiGrpc>("PushGrafanaLokiGrpc"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<PushGrafanaLokiGrpc>("PushGrafanaLokiGrpc"));
auto push_grafana_loki_grpc = test_controller.getProcessor();
CHECK(test_controller.plan->setProperty(push_grafana_loki_grpc, PushGrafanaLokiGrpc::Url, "localhost:10991"));
CHECK(test_controller.plan->setProperty(push_grafana_loki_grpc, PushGrafanaLokiGrpc::StreamLabels, "job=minifi,directory=/opt/minifi/logs/"));
@@ -60,8 +60,8 @@ class PushGrafanaLokiGrpcTestFixture {
public:
PushGrafanaLokiGrpcTestFixture()
: mock_loki_("10991"),
- test_controller_(std::make_unique<PushGrafanaLokiGrpc>("PushGrafanaLokiGrpc")),
- push_grafana_loki_grpc_(test_controller_.getProcessor<PushGrafanaLokiGrpc>()) {
+ test_controller_(minifi::test::utils::make_processor<PushGrafanaLokiGrpc>("PushGrafanaLokiGrpc")),
+ push_grafana_loki_grpc_(test_controller_.getProcessor()) {
LogTestController::getInstance().setDebug<TestPlan>();
LogTestController::getInstance().setDebug<minifi::core::Processor>();
LogTestController::getInstance().setTrace<minifi::core::ProcessSession>();
@@ -111,7 +111,7 @@ class PushGrafanaLokiGrpcTestFixture {
protected:
MockGrafanaLokiGrpc mock_loki_;
minifi::test::SingleProcessorTestController test_controller_;
- PushGrafanaLokiGrpc* push_grafana_loki_grpc_;
+ TypedProcessorWrapper<PushGrafanaLokiGrpc> push_grafana_loki_grpc_;
};
TEST_CASE_METHOD(PushGrafanaLokiGrpcTestFixture, "PushGrafanaLokiGrpc should send 1 log line to Grafana Loki in a trigger", "[PushGrafanaLokiGrpc]") {
diff --git a/extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp b/extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp
index 952eb6b0e..7dd9862b1 100644
--- a/extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp
+++ b/extensions/grafana-loki/tests/PushGrafanaLokiRESTTest.cpp
@@ -30,7 +30,7 @@
namespace org::apache::nifi::minifi::extensions::grafana::loki::test {
TEST_CASE("Url property is required", "[PushGrafanaLokiREST]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<PushGrafanaLokiREST>("PushGrafanaLokiREST"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<PushGrafanaLokiREST>("PushGrafanaLokiREST"));
auto push_grafana_loki_rest = test_controller.getProcessor();
test_controller.plan->setProperty(push_grafana_loki_rest, PushGrafanaLokiREST::Url, "");
@@ -40,7 +40,7 @@ TEST_CASE("Url property is required", "[PushGrafanaLokiREST]") {
}
TEST_CASE("Valid stream labels need to be set", "[PushGrafanaLokiREST]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<PushGrafanaLokiREST>("PushGrafanaLokiREST"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<PushGrafanaLokiREST>("PushGrafanaLokiREST"));
auto push_grafana_loki_rest = test_controller.getProcessor();
test_controller.plan->setProperty(push_grafana_loki_rest, PushGrafanaLokiREST::Url, "localhost:10990");
test_controller.plan->setProperty(push_grafana_loki_rest, PushGrafanaLokiREST::LogLineBatchSize, "1");
@@ -54,7 +54,7 @@ TEST_CASE("Valid stream labels need to be set", "[PushGrafanaLokiREST]") {
}
TEST_CASE("Log Line Batch Size cannot be 0", "[PushGrafanaLokiREST]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<PushGrafanaLokiREST>("PushGrafanaLokiREST"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<PushGrafanaLokiREST>("PushGrafanaLokiREST"));
auto push_grafana_loki_rest = test_controller.getProcessor();
test_controller.plan->setProperty(push_grafana_loki_rest, PushGrafanaLokiREST::Url, "localhost:10990");
test_controller.plan->setProperty(push_grafana_loki_rest, PushGrafanaLokiREST::StreamLabels, "job=minifi,directory=/opt/minifi/logs/");
@@ -66,8 +66,8 @@ class PushGrafanaLokiRESTTestFixture {
public:
PushGrafanaLokiRESTTestFixture()
: mock_loki_("10990"),
- test_controller_(std::make_unique<PushGrafanaLokiREST>("PushGrafanaLokiREST")),
- push_grafana_loki_rest_(test_controller_.getProcessor<PushGrafanaLokiREST>()) {
+ test_controller_(minifi::test::utils::make_processor<PushGrafanaLokiREST>("PushGrafanaLokiREST")),
+ push_grafana_loki_rest_(test_controller_.getProcessor()) {
LogTestController::getInstance().setDebug<TestPlan>();
LogTestController::getInstance().setDebug<minifi::core::Processor>();
LogTestController::getInstance().setTrace<minifi::core::ProcessSession>();
@@ -161,7 +161,7 @@ class PushGrafanaLokiRESTTestFixture {
protected:
MockGrafanaLokiREST mock_loki_;
minifi::test::SingleProcessorTestController test_controller_;
- PushGrafanaLokiREST* push_grafana_loki_rest_;
+ TypedProcessorWrapper<PushGrafanaLokiREST> push_grafana_loki_rest_;
};
TEST_CASE_METHOD(PushGrafanaLokiRESTTestFixture, "PushGrafanaLokiREST should send 1 log line to Grafana Loki in a trigger", "[PushGrafanaLokiREST]") {
@@ -273,7 +273,7 @@ TEST_CASE_METHOD(PushGrafanaLokiRESTTestFixture, "PushGrafanaLokiREST should wai
}
TEST_CASE("If username is set, password is also required to be set", "[PushGrafanaLokiREST]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<PushGrafanaLokiREST>("PushGrafanaLokiREST"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<PushGrafanaLokiREST>("PushGrafanaLokiREST"));
auto push_grafana_loki_rest = test_controller.getProcessor();
test_controller.plan->setProperty(push_grafana_loki_rest, PushGrafanaLokiREST::Url, "localhost:10990");
test_controller.plan->setProperty(push_grafana_loki_rest, PushGrafanaLokiREST::StreamLabels, "job=minifi,directory=/opt/minifi/logs/");
diff --git a/extensions/kafka/ConsumeKafka.h b/extensions/kafka/ConsumeKafka.h
index e962de4bc..3a41ad4e1 100644
--- a/extensions/kafka/ConsumeKafka.h
+++ b/extensions/kafka/ConsumeKafka.h
@@ -286,8 +286,7 @@ class ConsumeKafka final : public KafkaProcessorBase {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit ConsumeKafka(std::string_view name, const utils::Identifier& uuid = utils::Identifier())
- : KafkaProcessorBase(name, uuid, core::logging::LoggerFactory<ConsumeKafka>::getLogger(uuid)) {}
+ using KafkaProcessorBase::KafkaProcessorBase;
ConsumeKafka(const ConsumeKafka&) = delete;
ConsumeKafka(ConsumeKafka&&) = delete;
diff --git a/extensions/kafka/KafkaProcessorBase.h b/extensions/kafka/KafkaProcessorBase.h
index 1d6a44c7c..457666d30 100644
--- a/extensions/kafka/KafkaProcessorBase.h
+++ b/extensions/kafka/KafkaProcessorBase.h
@@ -21,7 +21,7 @@
#include <string_view>
#include "controllers/SSLContextServiceInterface.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinitionBuilder.h"
#include "rdkafka_utils.h"
#include "utils/Enum.h"
@@ -88,10 +88,7 @@ class KafkaProcessorBase : public core::ProcessorImpl {
EXTENSIONAPI static constexpr auto Properties = std::to_array<core::PropertyReference>({SSLContextService,
SecurityProtocol, KerberosServiceName, KerberosPrincipal, KerberosKeytabPath, SASLMechanism, Username, Password});
- KafkaProcessorBase(
- const std::string_view name, const utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger)
- : core::ProcessorImpl(name, uuid),
- logger_(std::move(logger)) {}
+ using ProcessorImpl::ProcessorImpl;
KafkaProcessorBase(const KafkaProcessorBase&) = delete;
KafkaProcessorBase(KafkaProcessorBase&&) = delete;
@@ -104,7 +101,6 @@ class KafkaProcessorBase : public core::ProcessorImpl {
void setKafkaAuthenticationParameters(core::ProcessContext& context, gsl::not_null<rd_kafka_conf_t*> config);
kafka::SecurityProtocolOption security_protocol_{};
- std::shared_ptr<core::logging::Logger> logger_;
};
} // namespace org::apache::nifi::minifi::processors
diff --git a/extensions/kafka/PublishKafka.h b/extensions/kafka/PublishKafka.h
index 01f0bfef8..26ed6c3b3 100644
--- a/extensions/kafka/PublishKafka.h
+++ b/extensions/kafka/PublishKafka.h
@@ -247,8 +247,7 @@ class PublishKafka final : public KafkaProcessorBase {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PublishKafka(const std::string_view name, const utils::Identifier& uuid = {})
- : KafkaProcessorBase(name, uuid, core::logging::LoggerFactory<PublishKafka>::getLogger(uuid)) {}
+ using KafkaProcessorBase::KafkaProcessorBase;
PublishKafka(const PublishKafka&) = delete;
PublishKafka(PublishKafka&&) = delete;
diff --git a/extensions/kafka/tests/PublishKafkaTests.cpp b/extensions/kafka/tests/PublishKafkaTests.cpp
index a150fceb5..aeb286531 100644
--- a/extensions/kafka/tests/PublishKafkaTests.cpp
+++ b/extensions/kafka/tests/PublishKafkaTests.cpp
@@ -16,6 +16,7 @@
* limitations under the License.
*/
#include "unit/TestBase.h"
+#include "unit/TestUtils.h"
#include "unit/Catch.h"
#include "PublishKafka.h"
#include "unit/SingleProcessorTestController.h"
@@ -25,7 +26,7 @@ namespace org::apache::nifi::minifi::test {
TEST_CASE("Scheduling should fail when batch size is larger than the max queue message count", "[testPublishKafka]") {
LogTestController::getInstance().setTrace<TestPlan>();
LogTestController::getInstance().setTrace<processors::PublishKafka>();
- SingleProcessorTestController test_controller(std::make_unique<processors::PublishKafka>("PublishKafka"));
+ SingleProcessorTestController test_controller(minifi::test::utils::make_processor<processors::PublishKafka>("PublishKafka"));
const auto publish_kafka = test_controller.getProcessor();
REQUIRE(publish_kafka->setProperty(processors::PublishKafka::ClientName.name, "test_client"));
REQUIRE(publish_kafka->setProperty(processors::PublishKafka::SeedBrokers.name, "test_seedbroker"));
@@ -36,34 +37,34 @@ TEST_CASE("Scheduling should fail when batch size is larger than the max queue m
TEST_CASE("Compress Codec property") {
using processors::PublishKafka;
- SingleProcessorTestController test_controller(std::make_unique<PublishKafka>("PublishKafka"));
+ SingleProcessorTestController test_controller(minifi::test::utils::make_processor<PublishKafka>("PublishKafka"));
REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::ClientName.name, "test_client"));
REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::SeedBrokers.name, "test_seedbroker"));
REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::Topic.name, "test_topic"));
REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::MessageTimeOut.name, "10ms"));
SECTION("none") {
- REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::CompressCodec.name, "none"));
+ REQUIRE(test_controller.getProcessor()->setProperty(PublishKafka::CompressCodec.name, "none"));
REQUIRE_NOTHROW(test_controller.trigger("input"));
}
SECTION("gzip") {
- REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::CompressCodec.name, "gzip"));
+ REQUIRE(test_controller.getProcessor()->setProperty(PublishKafka::CompressCodec.name, "gzip"));
REQUIRE_NOTHROW(test_controller.trigger("input"));
}
SECTION("snappy") {
- REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::CompressCodec.name, "snappy"));
+ REQUIRE(test_controller.getProcessor()->setProperty(PublishKafka::CompressCodec.name, "snappy"));
REQUIRE_NOTHROW(test_controller.trigger("input"));
}
SECTION("lz4") {
- REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::CompressCodec.name, "lz4"));
+ REQUIRE(test_controller.getProcessor()->setProperty(PublishKafka::CompressCodec.name, "lz4"));
REQUIRE_NOTHROW(test_controller.trigger("input"));
}
SECTION("zstd") {
- REQUIRE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::CompressCodec.name, "zstd"));
+ REQUIRE(test_controller.getProcessor()->setProperty(PublishKafka::CompressCodec.name, "zstd"));
REQUIRE_NOTHROW(test_controller.trigger("input"));
}
SECTION("foo") {
- REQUIRE_FALSE(test_controller.getProcessor<PublishKafka>()->setProperty(PublishKafka::CompressCodec.name, "foo"));
+ REQUIRE_FALSE(test_controller.getProcessor()->setProperty(PublishKafka::CompressCodec.name, "foo"));
}
}
diff --git a/extensions/libarchive/BinFiles.cpp b/extensions/libarchive/BinFiles.cpp
index b9366d6d6..a82ab464c 100644
--- a/extensions/libarchive/BinFiles.cpp
+++ b/extensions/libarchive/BinFiles.cpp
@@ -290,7 +290,7 @@ void BinFiles::onTrigger(core::ProcessContext& context, core::ProcessSession& se
const bool valid_batch = assumeOwnershipOfNextBatch(session);
if (auto ready_bins = gatherReadyBins(context); ready_bins.empty()) {
if (!valid_batch) {
- yield();
+ context.yield();
}
} else {
processReadyBins(std::move(ready_bins), session);
diff --git a/extensions/libarchive/CompressContent.h b/extensions/libarchive/CompressContent.h
index a1a2666b1..361ca871c 100644
--- a/extensions/libarchive/CompressContent.h
+++ b/extensions/libarchive/CompressContent.h
@@ -85,10 +85,7 @@ namespace org::apache::nifi::minifi::processors {
class CompressContent : public core::ProcessorImpl {
public:
- explicit CompressContent(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<CompressContent>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~CompressContent() override = default;
EXTENSIONAPI static constexpr const char* Description = "Compresses or decompresses the contents of FlowFiles using a user-specified compression algorithm "
diff --git a/extensions/libarchive/FocusArchiveEntry.cpp b/extensions/libarchive/FocusArchiveEntry.cpp
index 6ee446ade..58cb90c23 100644
--- a/extensions/libarchive/FocusArchiveEntry.cpp
+++ b/extensions/libarchive/FocusArchiveEntry.cpp
@@ -64,7 +64,7 @@ void FocusArchiveEntry::onTrigger(core::ProcessContext& context, core::ProcessSe
archiveMetadata.focusedEntry = context.getProperty(Path).value_or("");
flowFile->getAttribute("filename", archiveMetadata.archiveName);
- session.read(flowFile, ReadCallback{this, &file_man, &archiveMetadata});
+ session.read(flowFile, ReadCallback{&context, &file_man, &archiveMetadata});
// For each extracted entry, import & stash to key
std::string targetEntryStashKey;
@@ -131,7 +131,7 @@ void FocusArchiveEntry::onTrigger(core::ProcessContext& context, core::ProcessSe
struct FocusArchiveEntryReadData {
std::shared_ptr<io::InputStream> stream;
- core::Processor *processor = nullptr;
+ core::ProcessContext* context = nullptr;
std::array<std::byte, BUFFER_SIZE> buf{};
};
@@ -145,9 +145,9 @@ la_ssize_t FocusArchiveEntry::ReadCallback::read_cb(struct archive * a, void *d,
do {
last_read = data->stream->read(data->buf);
read += last_read;
- } while (data->processor->isRunning() && last_read > 0 && !io::isError(last_read) && read < BUFFER_SIZE);
+ } while (data->context->isRunning() && last_read > 0 && !io::isError(last_read) && read < BUFFER_SIZE);
- if (!data->processor->isRunning()) {
+ if (!data->context->isRunning()) {
archive_set_error(a, EINTR, "Processor shut down during read");
return -1;
}
@@ -162,7 +162,7 @@ int64_t FocusArchiveEntry::ReadCallback::operator()(const std::shared_ptr<io::In
FocusArchiveEntryReadData data;
data.stream = stream;
- data.processor = proc_;
+ data.context = context_;
archive_read_support_format_all(input_archive.get());
archive_read_support_filter_all(input_archive.get());
@@ -173,7 +173,7 @@ int64_t FocusArchiveEntry::ReadCallback::operator()(const std::shared_ptr<io::In
return nlen;
}
- while (proc_->isRunning()) {
+ while (context_->isRunning()) {
auto res = archive_read_next_header(input_archive.get(), &entry);
if (res == ARCHIVE_EOF) {
@@ -236,9 +236,9 @@ int64_t FocusArchiveEntry::ReadCallback::operator()(const std::shared_ptr<io::In
return nlen;
}
-FocusArchiveEntry::ReadCallback::ReadCallback(core::Processor *processor, utils::file::FileManager *file_man, ArchiveMetadata *archiveMetadata)
+FocusArchiveEntry::ReadCallback::ReadCallback(core::ProcessContext* context, utils::file::FileManager *file_man, ArchiveMetadata *archiveMetadata)
: file_man_(file_man),
- proc_(processor),
+ context_(context),
_archiveMetadata(archiveMetadata) {
}
diff --git a/extensions/libarchive/FocusArchiveEntry.h b/extensions/libarchive/FocusArchiveEntry.h
index 89be2362c..23ea6f61f 100644
--- a/extensions/libarchive/FocusArchiveEntry.h
+++ b/extensions/libarchive/FocusArchiveEntry.h
@@ -24,7 +24,7 @@
#include "ArchiveMetadata.h"
#include "FlowFileRecord.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -38,10 +38,7 @@ namespace org::apache::nifi::minifi::processors {
class FocusArchiveEntry : public core::ProcessorImpl {
public:
- explicit FocusArchiveEntry(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<FocusArchiveEntry>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~FocusArchiveEntry() override = default;
EXTENSIONAPI static constexpr const char* Description = "Allows manipulation of entries within an archive (e.g. TAR) by focusing on one entry within the archive at a time. "
@@ -68,12 +65,12 @@ class FocusArchiveEntry : public core::ProcessorImpl {
class ReadCallback {
public:
- explicit ReadCallback(core::Processor*, utils::file::FileManager *file_man, ArchiveMetadata *archiveMetadata);
+ explicit ReadCallback(core::ProcessContext* context, utils::file::FileManager *file_man, ArchiveMetadata *archiveMetadata);
int64_t operator()(const std::shared_ptr<io::InputStream>& stream) const;
private:
utils::file::FileManager *file_man_;
- core::Processor * const proc_;
+ core::ProcessContext* const context_;
std::shared_ptr<core::logging::Logger> logger_ = core::logging::LoggerFactory<FocusArchiveEntry>::getLogger();
ArchiveMetadata *_archiveMetadata;
static int ok_cb(struct archive *, void* /*d*/) { return ARCHIVE_OK; }
diff --git a/extensions/libarchive/ManipulateArchive.cpp b/extensions/libarchive/ManipulateArchive.cpp
index 0193fe749..e358e7b65 100644
--- a/extensions/libarchive/ManipulateArchive.cpp
+++ b/extensions/libarchive/ManipulateArchive.cpp
@@ -92,7 +92,7 @@ void ManipulateArchive::onSchedule(core::ProcessContext& context, core::ProcessS
}
}
-void ManipulateArchive::onTrigger(core::ProcessContext&, core::ProcessSession& session) {
+void ManipulateArchive::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
std::shared_ptr<core::FlowFile> flowFile = session.get();
if (!flowFile) {
@@ -102,7 +102,7 @@ void ManipulateArchive::onTrigger(core::ProcessContext&, core::ProcessSession& s
ArchiveMetadata archiveMetadata;
utils::file::FileManager file_man;
- session.read(flowFile, FocusArchiveEntry::ReadCallback{this, &file_man, &archiveMetadata});
+ session.read(flowFile, FocusArchiveEntry::ReadCallback{&context, &file_man, &archiveMetadata});
auto entries_end = archiveMetadata.entryMetadata.end();
diff --git a/extensions/libarchive/ManipulateArchive.h b/extensions/libarchive/ManipulateArchive.h
index 769f6399f..1a6d897f8 100644
--- a/extensions/libarchive/ManipulateArchive.h
+++ b/extensions/libarchive/ManipulateArchive.h
@@ -25,7 +25,7 @@
#include "FlowFileRecord.h"
#include "ArchiveMetadata.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -38,10 +38,7 @@ using core::logging::Logger;
class ManipulateArchive : public core::ProcessorImpl {
public:
- explicit ManipulateArchive(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ManipulateArchive>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~ManipulateArchive() override = default;
EXTENSIONAPI static constexpr const char* Description = "Performs an operation which manipulates an archive without needing to split the archive into multiple FlowFiles.";
diff --git a/extensions/libarchive/MergeContent.h b/extensions/libarchive/MergeContent.h
index 398eea6cf..0dbed6f40 100644
--- a/extensions/libarchive/MergeContent.h
+++ b/extensions/libarchive/MergeContent.h
@@ -294,9 +294,8 @@ class KeepAllUniqueAttributesMerger: public AttributeMerger {
*/
class MergeContent : public processors::BinFiles {
public:
- explicit MergeContent(const std::string& name, const utils::Identifier& uuid = {})
- : processors::BinFiles(name, uuid) {
- logger_ = core::logging::LoggerFactory<MergeContent>::getLogger(uuid_);
+ explicit MergeContent(core::ProcessorMetadata info)
+ : processors::BinFiles(info) {
mergeStrategy_ = merge_content_options::MERGE_STRATEGY_DEFRAGMENT;
mergeFormat_ = merge_content_options::MERGE_FORMAT_CONCAT_VALUE;
delimiterStrategy_ = merge_content_options::DELIMITER_STRATEGY_FILENAME;
diff --git a/extensions/libarchive/UnfocusArchiveEntry.h b/extensions/libarchive/UnfocusArchiveEntry.h
index fa662b6fb..f89832d5b 100644
--- a/extensions/libarchive/UnfocusArchiveEntry.h
+++ b/extensions/libarchive/UnfocusArchiveEntry.h
@@ -40,10 +40,7 @@ using core::logging::Logger;
class UnfocusArchiveEntry : public core::ProcessorImpl {
public:
- explicit UnfocusArchiveEntry(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<UnfocusArchiveEntry>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~UnfocusArchiveEntry() override = default;
EXTENSIONAPI static constexpr const char* Description = "Restores a FlowFile which has had an archive entry focused via FocusArchiveEntry to its original state.";
diff --git a/extensions/libarchive/tests/CompressContentTests.cpp b/extensions/libarchive/tests/CompressContentTests.cpp
index de1f94ef7..157d75465 100644
--- a/extensions/libarchive/tests/CompressContentTests.cpp
+++ b/extensions/libarchive/tests/CompressContentTests.cpp
@@ -117,7 +117,7 @@ class CompressDecompressionTestController : public TestController {
std::shared_ptr<TestRepository> repo = std::make_shared<TestRepository>();
- processor = std::make_shared<minifi::processors::CompressContent>("compresscontent");
+ processor = minifi::test::utils::make_processor<minifi::processors::CompressContent>("compresscontent");
processor->initialize();
utils::Identifier processoruuid = processor->getUUID();
REQUIRE(processoruuid);
diff --git a/extensions/libarchive/tests/FocusArchiveTests.cpp b/extensions/libarchive/tests/FocusArchiveTests.cpp
index f455c897a..7915fd169 100644
--- a/extensions/libarchive/tests/FocusArchiveTests.cpp
+++ b/extensions/libarchive/tests/FocusArchiveTests.cpp
@@ -36,6 +36,7 @@
#include "unit/Catch.h"
#include "repository/VolatileContentRepository.h"
#include "unit/ProvenanceTestHelper.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::processors::test {
@@ -49,13 +50,13 @@ const char* FOCUSED_CONTENT = FILE_CONTENT[0];
TEST_CASE("Test Creation of FocusArchiveEntry", "[getfileCreate]") {
TestController testController;
- auto processor = std::make_shared<FocusArchiveEntry>("processorname");
+ auto processor = minifi::test::utils::make_processor<FocusArchiveEntry>("processorname");
REQUIRE(processor->getName() == "processorname");
}
TEST_CASE("Test Creation of UnfocusArchiveEntry", "[getfileCreate]") {
TestController testController;
- auto processor = std::make_shared<UnfocusArchiveEntry>("processorname");
+ auto processor = minifi::test::utils::make_processor<UnfocusArchiveEntry>("processorname");
REQUIRE(processor->getName() == "processorname");
REQUIRE(processor->getUUID());
}
diff --git a/extensions/libarchive/tests/ManipulateArchiveTests.cpp b/extensions/libarchive/tests/ManipulateArchiveTests.cpp
index e7c5a6b62..55aeebdba 100644
--- a/extensions/libarchive/tests/ManipulateArchiveTests.cpp
+++ b/extensions/libarchive/tests/ManipulateArchiveTests.cpp
@@ -22,6 +22,7 @@
#include <utility>
#include "unit/TestBase.h"
+#include "unit/TestUtils.h"
#include "unit/Catch.h"
#include "util/ArchiveTests.h"
#include "core/PropertyDefinition.h"
@@ -110,7 +111,7 @@ bool run_archive_test(TAE_MAP_T input_map, TAE_MAP_T output_map, const PROP_MAP_
TEST_CASE("Test creation of ManipulateArchive", "[manipulatearchiveCreate]") {
TestController testController;
- auto processor = std::make_shared<ManipulateArchive>("processorname");
+ auto processor = minifi::test::utils::make_processor<ManipulateArchive>("processorname");
REQUIRE(processor->getName() == "processorname");
REQUIRE(processor->getUUID());
}
diff --git a/extensions/libarchive/tests/MergeFileTests.cpp b/extensions/libarchive/tests/MergeFileTests.cpp
index cf45fa733..158d466c7 100644
--- a/extensions/libarchive/tests/MergeFileTests.cpp
+++ b/extensions/libarchive/tests/MergeFileTests.cpp
@@ -163,11 +163,11 @@ class MergeTestController : public TestController {
auto content_repo = std::make_shared<core::repository::VolatileContentRepository>();
content_repo->initialize(std::make_shared<minifi::ConfigureImpl>());
- merge_content_processor_ = std::make_unique<minifi::processors::MergeContent>("mergecontent");
+ merge_content_processor_ = minifi::test::utils::make_processor<minifi::processors::MergeContent>("mergecontent");
merge_content_processor_->initialize();
utils::Identifier processoruuid = merge_content_processor_->getUUID();
REQUIRE(processoruuid);
- log_attribute_processor_ = std::make_unique<minifi::processors::LogAttribute>("logattribute");
+ log_attribute_processor_ = minifi::test::utils::make_processor<minifi::processors::LogAttribute>("logattribute");
utils::Identifier logAttributeuuid = log_attribute_processor_->getUUID();
REQUIRE(logAttributeuuid);
@@ -903,7 +903,7 @@ TEST_CASE_METHOD(MergeTestController, "Maximum Group Size is respected", "[testM
}
TEST_CASE("Empty MergeContent yields") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<minifi::processors::MergeContent>("mergeContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::MergeContent>("mergeContent")};
controller.trigger();
auto merge_content = controller.getProcessor();
@@ -911,7 +911,7 @@ TEST_CASE("Empty MergeContent yields") {
}
TEST_CASE("Empty MergeContent doesnt yield when processing readybins") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<minifi::processors::MergeContent>("mergeContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::MergeContent>("mergeContent")};
const auto merge_content = controller.getProcessor();
REQUIRE(controller.plan->setProperty(merge_content, minifi::processors::MergeContent::MaxBinAge, "100ms"));
REQUIRE(controller.plan->setProperty(merge_content, minifi::processors::MergeContent::MinEntries, "2"));
diff --git a/extensions/lua/tests/TestExecuteScriptProcessorWithLuaScript.cpp b/extensions/lua/tests/TestExecuteScriptProcessorWithLuaScript.cpp
index 9e71544f0..af216a2e9 100644
--- a/extensions/lua/tests/TestExecuteScriptProcessorWithLuaScript.cpp
+++ b/extensions/lua/tests/TestExecuteScriptProcessorWithLuaScript.cpp
@@ -22,6 +22,7 @@
#include "unit/SingleProcessorTestController.h"
#include "unit/TestBase.h"
+#include "unit/TestUtils.h"
#include "unit/Catch.h"
#include "../../script/ExecuteScript.h"
@@ -31,7 +32,7 @@
namespace org::apache::nifi::minifi::processors::test {
TEST_CASE("Lua: hello world") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
REQUIRE(execute_script->setProperty(ExecuteScript::ScriptEngine.name, "lua"));
@@ -81,7 +82,7 @@ end
}
TEST_CASE("Lua: Test session get should return None if there are no flowfiles in the incoming connections") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -103,7 +104,7 @@ end
TEST_CASE("Lua: Test Log", "[executescriptLuaLog]") {
LogTestController::getInstance().reset();
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -122,7 +123,7 @@ end
}
TEST_CASE("Lua: Test Read File", "[executescriptLuaRead]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -153,7 +154,7 @@ end
}
TEST_CASE("Lua: Test Write File", "[executescriptLuaWrite]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -186,7 +187,7 @@ TEST_CASE("Lua: Test Write File", "[executescriptLuaWrite]") {
TEST_CASE("Lua: Test Create", "[executescriptLuaCreate]") {
LogTestController::getInstance().reset();
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -211,7 +212,7 @@ end
TEST_CASE("Lua: Test Update Attribute", "[executescriptLuaUpdateAttribute]") {
LogTestController::getInstance().reset();
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -274,7 +275,7 @@ TEST_CASE("Lua: Test Require", "[executescriptLuaRequire]") {
TEST_CASE("Lua: Test Module Directory property", "[executescriptLuaModuleDirectoryProperty]") {
using org::apache::nifi::minifi::utils::file::get_executable_dir;
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -292,7 +293,7 @@ TEST_CASE("Lua: Test Module Directory property", "[executescriptLuaModuleDirecto
}
TEST_CASE("Lua: Non existent script file should throw", "[executescriptLuaNonExistentScriptFile]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -303,7 +304,7 @@ TEST_CASE("Lua: Non existent script file should throw", "[executescriptLuaNonExi
}
TEST_CASE("Lua can remove flowfiles", "[ExecuteScript]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
REQUIRE(execute_script->setProperty(ExecuteScript::ScriptEngine.name, "lua"));
@@ -320,7 +321,7 @@ TEST_CASE("Lua can remove flowfiles", "[ExecuteScript]") {
}
TEST_CASE("Lua can store states in StateManager", "[ExecuteScript]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<minifi::processors::ExecuteScript>();
REQUIRE(execute_script->setProperty(ExecuteScript::ScriptEngine.name, "lua"));
diff --git a/extensions/mqtt/processors/AbstractMQTTProcessor.cpp b/extensions/mqtt/processors/AbstractMQTTProcessor.cpp
index d5b50b52f..4d132baf1 100644
--- a/extensions/mqtt/processors/AbstractMQTTProcessor.cpp
+++ b/extensions/mqtt/processors/AbstractMQTTProcessor.cpp
@@ -107,7 +107,7 @@ void AbstractMQTTProcessor::onSchedule(core::ProcessContext& context, core::Proc
}
readProperties(context);
- checkProperties();
+ checkProperties(context);
initializeClient();
}
@@ -251,7 +251,7 @@ void AbstractMQTTProcessor::onTrigger(core::ProcessContext& context, core::Proce
if (!MQTTAsync_isConnected(client_)) {
logger_->log_error("Could not work with MQTT broker because disconnected to {}", uri_);
- yield();
+ context.yield();
return;
}
diff --git a/extensions/mqtt/processors/AbstractMQTTProcessor.h b/extensions/mqtt/processors/AbstractMQTTProcessor.h
index 5eea11b4a..8fdc4f045 100644
--- a/extensions/mqtt/processors/AbstractMQTTProcessor.h
+++ b/extensions/mqtt/processors/AbstractMQTTProcessor.h
@@ -24,7 +24,7 @@
#include <shared_mutex>
#include "core/PropertyDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/Core.h"
@@ -86,9 +86,7 @@ static constexpr const char* const MQTT_SECURITY_PROTOCOL_SSL = "ssl";
class AbstractMQTTProcessor : public core::ProcessorImpl {
public:
- explicit AbstractMQTTProcessor(std::string_view name, const utils::Identifier& uuid = {}, std::shared_ptr<core::ProcessorMetrics> metrics = {})
- : core::ProcessorImpl(name, uuid, std::move(metrics)) {
- }
+ using ProcessorImpl::ProcessorImpl;
~AbstractMQTTProcessor() override {
freeResources();
@@ -221,7 +219,7 @@ class AbstractMQTTProcessor : public core::ProcessorImpl {
/**
* Checks property consistency before connecting to broker
*/
- virtual void checkProperties() {
+ virtual void checkProperties(core::ProcessContext& /*context*/) {
}
/**
diff --git a/extensions/mqtt/processors/ConsumeMQTT.cpp b/extensions/mqtt/processors/ConsumeMQTT.cpp
index 9182a3969..b354eef22 100644
--- a/extensions/mqtt/processors/ConsumeMQTT.cpp
+++ b/extensions/mqtt/processors/ConsumeMQTT.cpp
@@ -204,9 +204,9 @@ void ConsumeMQTT::resolveTopicFromAlias(SmartMessage& smart_message) {
}
-void ConsumeMQTT::checkProperties() {
- auto is_property_explicitly_set = [this](const std::string_view property_name) -> bool {
- const auto property_values = getAllPropertyValues(property_name) | utils::orThrow("It should only be called on valid property");
+void ConsumeMQTT::checkProperties(core::ProcessContext& context) {
+ auto is_property_explicitly_set = [&context](const std::string_view property_name) -> bool {
+ const auto property_values = context.getAllPropertyValues(property_name) | utils::orThrow("It should only be called on valid property");
return !property_values.empty();
};
if (mqtt_version_ == mqtt::MqttVersions::V_3_1_0 || mqtt_version_ == mqtt::MqttVersions::V_3_1_1 || mqtt_version_ == mqtt::MqttVersions::V_3X_AUTO) {
diff --git a/extensions/mqtt/processors/ConsumeMQTT.h b/extensions/mqtt/processors/ConsumeMQTT.h
index 2f9ba7eff..87c2b3d31 100644
--- a/extensions/mqtt/processors/ConsumeMQTT.h
+++ b/extensions/mqtt/processors/ConsumeMQTT.h
@@ -39,10 +39,7 @@ namespace org::apache::nifi::minifi::processors {
class ConsumeMQTT : public processors::AbstractMQTTProcessor {
public:
- explicit ConsumeMQTT(std::string_view name, const utils::Identifier& uuid = {})
- : processors::AbstractMQTTProcessor(name, uuid) {
- logger_ = core::logging::LoggerFactory<ConsumeMQTT>::getLogger(uuid_);
- }
+ using AbstractMQTTProcessor::AbstractMQTTProcessor;
EXTENSIONAPI static constexpr const char* Description = "This Processor gets the contents of a FlowFile from a MQTT broker for a specified topic. "
"The the payload of the MQTT message becomes content of a FlowFile";
@@ -163,7 +160,7 @@ class ConsumeMQTT : public processors::AbstractMQTTProcessor {
*/
void startupClient() override;
- void checkProperties() override;
+ void checkProperties(core::ProcessContext& context) override;
void checkBrokerLimitsImpl() override;
/**
diff --git a/extensions/mqtt/processors/PublishMQTT.cpp b/extensions/mqtt/processors/PublishMQTT.cpp
index 9a93e38cd..55f885b4f 100644
--- a/extensions/mqtt/processors/PublishMQTT.cpp
+++ b/extensions/mqtt/processors/PublishMQTT.cpp
@@ -59,7 +59,7 @@ void PublishMQTT::onTriggerImpl(core::ProcessContext& context, core::ProcessSess
std::shared_ptr<core::FlowFile> flow_file = session.get();
if (!flow_file) {
- yield();
+ context.yield();
return;
}
@@ -132,9 +132,9 @@ bool PublishMQTT::sendMessage(const std::vector<std::byte>& buffer, const std::s
return send_finished_task.get_future().get();
}
-void PublishMQTT::checkProperties() {
- auto is_property_explicitly_set = [this](const std::string_view property_name) -> bool {
- const auto property_values = getAllPropertyValues(property_name) | utils::orThrow("It should only be called on valid property");
+void PublishMQTT::checkProperties(core::ProcessContext& context) {
+ auto is_property_explicitly_set = [&context](const std::string_view property_name) -> bool {
+ const auto property_values = context.getAllPropertyValues(property_name) | utils::orThrow("It should only be called on valid property");
return !property_values.empty();
};
if ((mqtt_version_ == mqtt::MqttVersions::V_3_1_0 || mqtt_version_ == mqtt::MqttVersions::V_3_1_1 || mqtt_version_ == mqtt::MqttVersions::V_3X_AUTO)) {
@@ -293,7 +293,7 @@ uint16_t PublishMQTT::InFlightMessageCounter::getCounter() const {
return counter_;
}
-PublishMQTT::PublishMQTTMetrics::PublishMQTTMetrics(const core::Processor& source_processor, const InFlightMessageCounter& in_flight_message_counter)
+PublishMQTT::PublishMQTTMetrics::PublishMQTTMetrics(const core::ProcessorImpl& source_processor, const InFlightMessageCounter& in_flight_message_counter)
: core::ProcessorMetricsImpl(source_processor),
in_flight_message_counter_(&in_flight_message_counter) {
}
diff --git a/extensions/mqtt/processors/PublishMQTT.h b/extensions/mqtt/processors/PublishMQTT.h
index 398216751..d6d9ed7cd 100644
--- a/extensions/mqtt/processors/PublishMQTT.h
+++ b/extensions/mqtt/processors/PublishMQTT.h
@@ -39,10 +39,9 @@ namespace org::apache::nifi::minifi::processors {
class PublishMQTT : public processors::AbstractMQTTProcessor {
public:
- explicit PublishMQTT(const std::string_view name, const utils::Identifier& uuid = {})
- : processors::AbstractMQTTProcessor(name, uuid) {
+ explicit PublishMQTT(core::ProcessorMetadata info)
+ : processors::AbstractMQTTProcessor(info) {
metrics_ = gsl::make_not_null(std::make_shared<PublishMQTTMetrics>(*this, in_flight_message_counter_));
- logger_ = core::logging::LoggerFactory<PublishMQTT>::getLogger(uuid_);
}
EXTENSIONAPI static constexpr const char* Description = "PublishMQTT serializes FlowFile content as an MQTT payload, sending the message to the configured topic and broker.";
@@ -111,7 +110,7 @@ class PublishMQTT : public processors::AbstractMQTTProcessor {
class PublishMQTTMetrics : public core::ProcessorMetricsImpl {
public:
- PublishMQTTMetrics(const core::Processor& source_processor, const InFlightMessageCounter& in_flight_message_counter);
+ PublishMQTTMetrics(const core::ProcessorImpl& source_processor, const InFlightMessageCounter& in_flight_message_counter);
std::vector<state::response::SerializedResponseNode> serialize() override;
std::vector<state::PublishedMetric> calculateMetrics() override;
@@ -186,7 +185,7 @@ class PublishMQTT : public processors::AbstractMQTTProcessor {
// there is no need to do anything like subscribe in the beginning
}
- void checkProperties() override;
+ void checkProperties(core::ProcessContext& context) override;
void checkBrokerLimitsImpl() override;
bool retain_ = false;
diff --git a/extensions/opc/include/fetchopc.h b/extensions/opc/include/fetchopc.h
index 0017a73f0..6b7f0e04a 100644
--- a/extensions/opc/include/fetchopc.h
+++ b/extensions/opc/include/fetchopc.h
@@ -70,10 +70,7 @@ namespace org::apache::nifi::minifi::processors {
class FetchOPCProcessor final : public BaseOPCProcessor {
public:
- explicit FetchOPCProcessor(std::string_view name, const utils::Identifier& uuid = {})
- : BaseOPCProcessor(name, uuid) {
- logger_ = core::logging::LoggerFactory<FetchOPCProcessor>::getLogger(uuid_);
- }
+ using BaseOPCProcessor::BaseOPCProcessor;
EXTENSIONAPI static constexpr const char* Description = "Fetches OPC-UA node";
diff --git a/extensions/opc/include/opcbase.h b/extensions/opc/include/opcbase.h
index 3db9d738b..a8e699f20 100644
--- a/extensions/opc/include/opcbase.h
+++ b/extensions/opc/include/opcbase.h
@@ -73,10 +73,7 @@ class BaseOPCProcessor : public core::ProcessorImpl {
PathReferenceTypes
});
-
- explicit BaseOPCProcessor(std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- }
+ using ProcessorImpl::ProcessorImpl;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& factory) override;
diff --git a/extensions/opc/include/putopc.h b/extensions/opc/include/putopc.h
index c4ecff7e9..4a64b2e29 100644
--- a/extensions/opc/include/putopc.h
+++ b/extensions/opc/include/putopc.h
@@ -112,10 +112,7 @@ class PutOPCProcessor final : public BaseOPCProcessor {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PutOPCProcessor(std::string_view name, const utils::Identifier& uuid = {})
- : BaseOPCProcessor(name, uuid) {
- logger_ = core::logging::LoggerFactory<PutOPCProcessor>::getLogger(uuid_);
- }
+ using BaseOPCProcessor::BaseOPCProcessor;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
diff --git a/extensions/opc/src/fetchopc.cpp b/extensions/opc/src/fetchopc.cpp
index 7ac232ac6..f148d2de4 100644
--- a/extensions/opc/src/fetchopc.cpp
+++ b/extensions/opc/src/fetchopc.cpp
@@ -67,7 +67,7 @@ void FetchOPCProcessor::onTrigger(core::ProcessContext& context, core::ProcessSe
logger_->log_trace("FetchOPCProcessor::onTrigger");
if (!reconnect()) {
- yield();
+ context.yield();
return;
}
@@ -92,7 +92,7 @@ void FetchOPCProcessor::onTrigger(core::ProcessContext& context, core::ProcessSe
my_id.identifier.string = UA_STRING_ALLOC(node_id_.c_str()); // NOLINT(cppcoreguidelines-pro-type-union-access)
} else {
logger_->log_error("Unhandled id type: '{}'. No flowfiles are generated.", magic_enum::enum_underlying(id_type_));
- yield();
+ context.yield();
return;
}
connection_->traverse(my_id, found_cb, "", max_depth_);
@@ -101,7 +101,7 @@ void FetchOPCProcessor::onTrigger(core::ProcessContext& context, core::ProcessSe
auto sc = connection_->translateBrowsePathsToNodeIdsRequest(node_id_, translated_node_ids_, namespace_idx_, path_reference_types_, logger_);
if (sc != UA_STATUSCODE_GOOD) {
logger_->log_error("Failed to translate {} to node id, no flow files will be generated ({})", node_id_.c_str(), UA_StatusCode_name(sc));
- yield();
+ context.yield();
return;
}
}
@@ -111,10 +111,10 @@ void FetchOPCProcessor::onTrigger(core::ProcessContext& context, core::ProcessSe
}
if (nodes_found == 0) {
logger_->log_warn("Connected to OPC server, but no variable nodes were found. Configuration might be incorrect! Yielding...");
- yield();
+ context.yield();
} else if (variables_found == 0) {
logger_->log_warn("Found no variables when traversing the specified node. No flowfiles are generated. Yielding...");
- yield();
+ context.yield();
}
state_manager_->set(state_map);
diff --git a/extensions/opencv/CaptureRTSPFrame.h b/extensions/opencv/CaptureRTSPFrame.h
index f2bcf7243..aa8d7434b 100644
--- a/extensions/opencv/CaptureRTSPFrame.h
+++ b/extensions/opencv/CaptureRTSPFrame.h
@@ -26,7 +26,7 @@
#include <opencv2/opencv.hpp>
#include "core/logging/LoggerFactory.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinitionBuilder.h"
#include "io/StreamPipe.h"
#include "utils/gsl.h"
@@ -36,10 +36,7 @@ namespace org::apache::nifi::minifi::processors {
class CaptureRTSPFrame final : public core::ProcessorImpl {
public:
- explicit CaptureRTSPFrame(const std::string_view name, const utils::Identifier &uuid = {})
- : ProcessorImpl(std::move(name), uuid) {
- logger_ = core::logging::LoggerFactory<CaptureRTSPFrame>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Captures a frame from the RTSP stream at specified intervals.";
diff --git a/extensions/opencv/MotionDetector.h b/extensions/opencv/MotionDetector.h
index d2b9036e8..eeb053126 100644
--- a/extensions/opencv/MotionDetector.h
+++ b/extensions/opencv/MotionDetector.h
@@ -23,7 +23,7 @@
#include <utility>
#include "core/logging/LoggerFactory.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -35,10 +35,7 @@ namespace org::apache::nifi::minifi::processors {
class MotionDetector final : public core::ProcessorImpl {
public:
- explicit MotionDetector(const std::string_view name, const utils::Identifier &uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<MotionDetector>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Detect motion from captured images.";
diff --git a/extensions/python/ExecutePythonProcessor.cpp b/extensions/python/ExecutePythonProcessor.cpp
index c63544830..bdc7f6bbb 100644
--- a/extensions/python/ExecutePythonProcessor.cpp
+++ b/extensions/python/ExecutePythonProcessor.cpp
@@ -31,12 +31,12 @@
#include "utils/StringUtils.h"
#include "utils/file/FileUtils.h"
#include "utils/ProcessorConfigUtils.h"
-#include "minifi-cpp/utils/PropertyErrors.h"
+#include "utils/PropertyErrors.h"
#include "minifi-cpp/core/ProcessorDescriptor.h"
namespace org::apache::nifi::minifi::extensions::python::processors {
-void ExecutePythonProcessor::initialize(core::ProcessorDescriptor& self) {
+void ExecutePythonProcessor::initializeScript() {
if (processor_initialized_) {
logger_->log_debug("Processor has already been initialized, returning...");
return;
@@ -52,13 +52,17 @@ void ExecutePythonProcessor::initialize(core::ProcessorDescriptor& self) {
// so that we can provide manifest of processor identity on C2
python_script_engine_ = createScriptEngine();
initalizeThroughScriptEngine();
+}
+void ExecutePythonProcessor::initialize() {
+ initializeScript();
std::vector<core::PropertyReference> all_properties{Properties.begin(), Properties.end()};
for (auto& python_prop : python_properties_) {
all_properties.push_back(python_prop.getReference());
}
- self.setSupportedProperties(all_properties);
- self.setSupportedRelationships(Relationships);
+ setSupportedProperties(all_properties);
+ setSupportedRelationships(Relationships);
+ logger_->log_debug("Processor has already been initialized, returning...");
}
void ExecutePythonProcessor::initalizeThroughScriptEngine() {
@@ -85,6 +89,9 @@ void ExecutePythonProcessor::initalizeThroughScriptEngine() {
void ExecutePythonProcessor::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& /*sessionFactory*/) {
context.addAutoTerminatedRelationship(Original);
if (!processor_initialized_) {
+ script_file_path_ = context.getProperty(ScriptFile.name).value_or("");
+ script_to_exec_ = context.getProperty(ScriptBody.name).value_or("");
+ module_directory_ = context.getProperty(ModuleDirectory.name).value_or("");
loadScript();
python_script_engine_ = createScriptEngine();
initalizeThroughScriptEngine();
@@ -112,9 +119,8 @@ void ExecutePythonProcessor::onTrigger(core::ProcessContext& context, core::Proc
}
void ExecutePythonProcessor::appendPathForImportModules() const {
- std::string module_directory = getProperty(ModuleDirectory.name).value_or("");
- if (!module_directory.empty()) {
- python_script_engine_->appendModulePaths(utils::string::splitAndTrimRemovingEmpty(module_directory, ",") | ranges::to<std::vector<std::filesystem::path>>());
+ if (!module_directory_.empty()) {
+ python_script_engine_->appendModulePaths(utils::string::splitAndTrimRemovingEmpty(module_directory_, ",") | ranges::to<std::vector<std::filesystem::path>>());
}
}
@@ -128,22 +134,18 @@ void ExecutePythonProcessor::loadScriptFromFile() {
}
void ExecutePythonProcessor::loadScript() {
- std::string script_file = getProperty(ScriptFile.name).value_or("");
- std::string script_body = getProperty(ScriptBody.name).value_or("");
- if (script_file.empty() && script_body.empty()) {
+ if (script_file_path_.empty() && script_to_exec_.empty()) {
throw std::runtime_error("Neither Script Body nor Script File is available to execute");
}
- if (!script_file.empty()) {
- if (!script_body.empty()) {
+ if (!script_file_path_.empty()) {
+ if (!script_to_exec_.empty()) {
throw std::runtime_error("Only one of Script File or Script Body may be used");
}
- script_file_path_ = script_file;
loadScriptFromFile();
last_script_write_time_ = utils::file::last_write_time(script_file_path_);
return;
}
- script_to_exec_ = script_body;
}
void ExecutePythonProcessor::reloadScriptIfUsingScriptFileProperty() {
diff --git a/extensions/python/ExecutePythonProcessor.h b/extensions/python/ExecutePythonProcessor.h
index caefb7b6a..f74ce8cc1 100644
--- a/extensions/python/ExecutePythonProcessor.h
+++ b/extensions/python/ExecutePythonProcessor.h
@@ -28,7 +28,7 @@
#include <filesystem>
#include "concurrentqueue.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -40,8 +40,8 @@ namespace org::apache::nifi::minifi::extensions::python::processors {
class ExecutePythonProcessor : public core::ProcessorImpl {
public:
- explicit ExecutePythonProcessor(std::string_view name, const utils::Identifier &uuid = {})
- : ProcessorImpl(name, uuid),
+ explicit ExecutePythonProcessor(core::ProcessorMetadata info)
+ : ProcessorImpl(info),
processor_initialized_(false),
python_dynamic_(false),
reload_on_script_change_(true) {
@@ -90,7 +90,8 @@ class ExecutePythonProcessor : public core::ProcessorImpl {
EXTENSIONAPI static constexpr bool IsSingleThreaded = true;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- void initialize(core::ProcessorDescriptor& self) override;
+ void initializeScript();
+ void initialize() override;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
@@ -138,15 +139,13 @@ class ExecutePythonProcessor : public core::ProcessorImpl {
qualified_module_name_ = qualified_module_name;
}
- std::map<std::string, core::Property, std::less<>> getSupportedProperties() const override;
- nonstd::expected<core::Property, std::error_code> getSupportedProperty(std::string_view name) const override;
+ void setScriptFilePath(std::string script_file_path) {
+ script_file_path_ = script_file_path;
+ }
std::vector<core::Relationship> getPythonRelationships() const;
void setLoggerCallback(const std::function<void(core::logging::LOG_LEVEL level, const std::string& message)>& callback) override;
- nonstd::expected<std::string, std::error_code> getProperty(std::string_view name) const override;
- nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value) override;
-
private:
mutable std::mutex python_properties_mutex_;
std::vector<core::Property> python_properties_;
@@ -166,6 +165,7 @@ class ExecutePythonProcessor : public core::ProcessorImpl {
std::optional<std::string> python_class_name_;
std::vector<std::filesystem::path> python_paths_;
std::string qualified_module_name_;
+ std::string module_directory_;
void appendPathForImportModules() const;
void loadScriptFromFile();
diff --git a/extensions/python/PythonCreator.h b/extensions/python/PythonCreator.h
index e86ddc526..b326d34dc 100644
--- a/extensions/python/PythonCreator.h
+++ b/extensions/python/PythonCreator.h
@@ -38,9 +38,39 @@
#include "range/v3/view/filter.hpp"
#include "PythonDependencyInstaller.h"
#include "utils/file/PathUtils.h"
+#include "minifi-cpp/core/ProcessorDescriptor.h"
namespace org::apache::nifi::minifi::extensions::python {
+class DummyProcessorDescriptor : public core::ProcessorDescriptor {
+ public:
+ ~DummyProcessorDescriptor() override = default;
+
+ void setSupportedRelationships(std::span<const core::RelationshipDefinition> relationships) override {}
+ void setSupportedProperties(std::span<const core::PropertyReference> properties) override {}
+};
+
+class DummyLogger : public core::logging::Logger {
+ public:
+ void set_max_log_size(int size) override {}
+ std::optional<std::string> get_id() override {
+ return std::nullopt;
+ }
+ void log_string(core::logging::LOG_LEVEL level, std::string str) override {}
+ bool should_log(core::logging::LOG_LEVEL level) override {
+ return false;
+ }
+ [[nodiscard]] virtual core::logging::LOG_LEVEL level() const override {
+ return core::logging::LOG_LEVEL::off;
+ }
+
+ int getMaxLogSize() override {
+ return 0;
+ }
+
+ ~DummyLogger() override = default;
+};
+
/**
* Can be used to load the python processors from NiFi properties.
*/
@@ -51,9 +81,10 @@ class PythonCreator : public minifi::core::CoreComponentImpl {
}
~PythonCreator() override {
- for (const auto& clazz : registered_classes_) {
+ for (const auto& [clazz, factory] : registered_classes_) {
core::getClassLoader().unregisterClass(clazz);
}
+ registered_classes_.clear();
}
void configure(const std::shared_ptr<Configure> &configuration) override {
@@ -79,20 +110,25 @@ class PythonCreator : public minifi::core::CoreComponentImpl {
const std::string qualified_module_name_prefix = "org.apache.nifi.minifi.processors.";
std::string qualified_module_name = full_name.substr(qualified_module_name_prefix.length());
+ std::unique_ptr<PythonObjectFactory> factory;
if (path.string().find("nifi_python_processors") != std::string::npos) {
auto utils_path = (std::filesystem::path("nifi_python_processors") / "utils").string();
if (path.string().find(utils_path) != std::string::npos) {
continue;
}
logger_->log_info("Registering NiFi python processor: {}", class_name);
- core::getClassLoader().registerClass(class_name, std::make_unique<PythonObjectFactory>(path.string(), script_name.string(),
- PythonProcessorType::NIFI_TYPE, std::vector<std::filesystem::path>{python_lib_path, std::filesystem::path{pathListings.value()}, path.parent_path()}, qualified_module_name));
+ factory = std::make_unique<PythonObjectFactory>(path.string(), script_name.string(),
+ PythonProcessorType::NIFI_TYPE, std::vector<std::filesystem::path>{python_lib_path, std::filesystem::path{pathListings.value()}, path.parent_path()}, qualified_module_name);
} else {
logger_->log_info("Registering MiNiFi python processor: {}", class_name);
- core::getClassLoader().registerClass(class_name, std::make_unique<PythonObjectFactory>(path.string(), script_name.string(),
- PythonProcessorType::MINIFI_TYPE, std::vector<std::filesystem::path>{python_lib_path, std::filesystem::path{pathListings.value()}}, qualified_module_name));
+ factory = std::make_unique<PythonObjectFactory>(path.string(), script_name.string(),
+ PythonProcessorType::MINIFI_TYPE, std::vector<std::filesystem::path>{python_lib_path, std::filesystem::path{pathListings.value()}}, qualified_module_name);
+ }
+ if (!registered_classes_.emplace(class_name, factory.get()).second) {
+ core::getClassLoader().registerClass(class_name, std::move(factory));
+ } else {
+ logger_->log_warn("Ignoring duplicate python processor at '{}' with class '{}'", path.string(), class_name);
}
- registered_classes_.push_back(class_name);
try {
registerScriptDescription(class_name, full_name, path, script_name.string());
} catch (const PythonScriptWarning &warning) {
@@ -105,12 +141,21 @@ class PythonCreator : public minifi::core::CoreComponentImpl {
private:
void registerScriptDescription(const std::string& class_name, const std::string& full_name, const std::filesystem::path& path, const std::string& script_name) {
- auto processor = core::ClassLoader::getDefaultClassLoader().instantiate<python::processors::ExecutePythonProcessor>(class_name, utils::IdGenerator::getIdGenerator()->generate());
+ std::unique_ptr<python::processors::ExecutePythonProcessor> processor;
+ auto factory_it = registered_classes_.find(class_name);
+ if (factory_it != registered_classes_.end()) {
+ processor = utils::dynamic_unique_cast<python::processors::ExecutePythonProcessor>(factory_it->second->create(core::ProcessorMetadata{
+ .uuid = utils::IdGenerator::getIdGenerator()->generate(),
+ .name = class_name,
+ .logger = std::make_shared<DummyLogger>()
+ }));
+ }
if (!processor) {
logger_->log_error("Couldn't instantiate '{}' python processor", class_name);
return;
}
- processor->initialize();
+ DummyProcessorDescriptor descriptor;
+ processor->core::ProcessorImpl::initialize(descriptor);
minifi::BundleDetails details;
details.artifact = path.filename().string();
details.version = processor->getVersion() && !processor->getVersion()->empty() ? *processor->getVersion() : minifi::AgentBuild::VERSION;
@@ -185,7 +230,7 @@ class PythonCreator : public minifi::core::CoreComponentImpl {
return python_lib_path;
}
- std::vector<std::string> registered_classes_;
+ std::unordered_map<std::string, PythonObjectFactory*> registered_classes_;
std::vector<std::filesystem::path> classpaths_;
std::shared_ptr<core::logging::Logger> logger_ = core::logging::LoggerFactory<PythonCreator>::getLogger();
diff --git a/extensions/python/PythonObjectFactory.h b/extensions/python/PythonObjectFactory.h
index 3635f5b11..4c75deb38 100644
--- a/extensions/python/PythonObjectFactory.h
+++ b/extensions/python/PythonObjectFactory.h
@@ -27,13 +27,14 @@
#include "core/ClassLoader.h"
#include "ExecutePythonProcessor.h"
#include "utils/StringUtils.h"
+#include "core/ProcessorFactory.h"
enum class PythonProcessorType {
MINIFI_TYPE,
NIFI_TYPE
};
-class PythonObjectFactory : public org::apache::nifi::minifi::core::DefaultObjectFactory<org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor> {
+class PythonObjectFactory : public org::apache::nifi::minifi::core::ProcessorFactoryImpl<org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor> {
public:
explicit PythonObjectFactory(std::string file, std::string class_name, PythonProcessorType python_processor_type,
const std::vector<std::filesystem::path>& python_paths, std::string qualified_module_name)
@@ -44,8 +45,8 @@ class PythonObjectFactory : public org::apache::nifi::minifi::core::DefaultObjec
qualified_module_name_(std::move(qualified_module_name)) {
}
- std::unique_ptr<org::apache::nifi::minifi::core::CoreComponent> create(const std::string &name) override {
- auto obj = DefaultObjectFactory::create(name);
+ std::unique_ptr<org::apache::nifi::minifi::core::ProcessorApi> create(org::apache::nifi::minifi::core::ProcessorMetadata info) override {
+ auto obj = ProcessorFactoryImpl::create(info);
auto ptr = org::apache::nifi::minifi::utils::dynamic_unique_cast<org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor>(std::move(obj));
if (ptr == nullptr) {
return nullptr;
@@ -55,59 +56,11 @@ class PythonObjectFactory : public org::apache::nifi::minifi::core::DefaultObjec
ptr->setPythonPaths(python_paths_);
}
ptr->setQualifiedModuleName(qualified_module_name_);
- ptr->initialize();
- if (!ptr->setProperty(org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor::ScriptFile.name, file_)) {
- return nullptr;
- }
+ ptr->setScriptFilePath(file_);
+ ptr->initializeScript();
return ptr;
}
- std::unique_ptr<org::apache::nifi::minifi::core::CoreComponent> create(const std::string &name, const org::apache::nifi::minifi::utils::Identifier &uuid) override {
- auto obj = DefaultObjectFactory::create(name, uuid);
- auto ptr = org::apache::nifi::minifi::utils::dynamic_unique_cast<org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor>(std::move(obj));
- if (ptr == nullptr) {
- return nullptr;
- }
- if (python_processor_type_ == PythonProcessorType::NIFI_TYPE) {
- ptr->setPythonClassName(class_name_);
- ptr->setPythonPaths(python_paths_);
- }
- ptr->setQualifiedModuleName(qualified_module_name_);
- ptr->initialize();
- if (!ptr->setProperty(org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor::ScriptFile.name, file_)) {
- return nullptr;
- }
- return ptr;
- }
-
- org::apache::nifi::minifi::core::CoreComponent* createRaw(const std::string &name) override {
- auto ptr = dynamic_cast<org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor*>(DefaultObjectFactory::createRaw(name));
- if (python_processor_type_ == PythonProcessorType::NIFI_TYPE) {
- ptr->setPythonClassName(class_name_);
- ptr->setPythonPaths(python_paths_);
- }
- ptr->setQualifiedModuleName(qualified_module_name_);
- ptr->initialize();
- if (!ptr->setProperty(org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor::ScriptFile.name, file_)) {
- return nullptr;
- }
- return dynamic_cast<org::apache::nifi::minifi::core::CoreComponent*>(ptr);
- }
-
- org::apache::nifi::minifi::core::CoreComponent* createRaw(const std::string &name, const org::apache::nifi::minifi::utils::Identifier &uuid) override {
- auto ptr = dynamic_cast<org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor*>(DefaultObjectFactory::createRaw(name, uuid));
- if (python_processor_type_ == PythonProcessorType::NIFI_TYPE) {
- ptr->setPythonClassName(class_name_);
- ptr->setPythonPaths(python_paths_);
- }
- ptr->setQualifiedModuleName(qualified_module_name_);
- ptr->initialize();
- if (!ptr->setProperty(org::apache::nifi::minifi::extensions::python::processors::ExecutePythonProcessor::ScriptFile.name, file_)) {
- return nullptr;
- }
- return dynamic_cast<org::apache::nifi::minifi::core::CoreComponent*>(ptr);
- }
-
private:
std::string file_;
std::string class_name_;
diff --git a/extensions/python/PythonProcessor.cpp b/extensions/python/PythonProcessor.cpp
index 9fd7c26a8..2416b3288 100644
--- a/extensions/python/PythonProcessor.cpp
+++ b/extensions/python/PythonProcessor.cpp
@@ -24,8 +24,8 @@ namespace org::apache::nifi::minifi::extensions::python {
namespace core = org::apache::nifi::minifi::core;
-PythonProcessor::PythonProcessor(core::Processor* proc) :
- processor_(dynamic_cast<python::processors::ExecutePythonProcessor*>(proc)) {
+PythonProcessor::PythonProcessor(python::processors::ExecutePythonProcessor* proc) :
+ processor_(proc) {
gsl_Expects(processor_);
}
diff --git a/extensions/python/PythonProcessor.h b/extensions/python/PythonProcessor.h
index 4333aeaa9..84915b213 100644
--- a/extensions/python/PythonProcessor.h
+++ b/extensions/python/PythonProcessor.h
@@ -34,7 +34,7 @@ class ExecutePythonProcessor;
class PythonProcessor {
public:
- explicit PythonProcessor(core::Processor* proc);
+ explicit PythonProcessor(python::processors::ExecutePythonProcessor* proc);
void setSupportsDynamicProperties();
diff --git a/extensions/python/PythonScriptEngine.cpp b/extensions/python/PythonScriptEngine.cpp
index e68ba022f..d7fda0397 100644
--- a/extensions/python/PythonScriptEngine.cpp
+++ b/extensions/python/PythonScriptEngine.cpp
@@ -72,7 +72,7 @@ void PythonScriptEngine::evalFile(const std::filesystem::path& file_name) {
}
}
-void PythonScriptEngine::onInitialize(core::Processor* proc) {
+void PythonScriptEngine::onInitialize(processors::ExecutePythonProcessor* proc) {
auto newproc = std::make_shared<python::PythonProcessor>(proc);
if (processor_instance_.get() != nullptr) {
callProcessorObjectMethod("onInitialize", std::weak_ptr(newproc));
@@ -81,7 +81,7 @@ void PythonScriptEngine::onInitialize(core::Processor* proc) {
}
}
-void PythonScriptEngine::describe(core::Processor* proc) {
+void PythonScriptEngine::describe(processors::ExecutePythonProcessor* proc) {
auto newproc = std::make_shared<python::PythonProcessor>(proc);
if (processor_instance_.get() != nullptr) {
callRequiredProcessorObjectMethod("describe", std::weak_ptr(newproc));
diff --git a/extensions/python/PythonScriptEngine.h b/extensions/python/PythonScriptEngine.h
index e7d4d6466..e29383aaa 100644
--- a/extensions/python/PythonScriptEngine.h
+++ b/extensions/python/PythonScriptEngine.h
@@ -39,6 +39,10 @@
namespace org::apache::nifi::minifi::extensions::python {
+namespace processors {
+class ExecutePythonProcessor;
+} // namespace processors
+
class PythonScriptEngine {
public:
PythonScriptEngine();
@@ -143,8 +147,8 @@ class PythonScriptEngine {
bindings_.put(name, value);
}
- void onInitialize(core::Processor* proc);
- void describe(core::Processor* proc);
+ void onInitialize(processors::ExecutePythonProcessor* proc);
+ void describe(processors::ExecutePythonProcessor* proc);
void onSchedule(core::ProcessContext& context);
void onTrigger(core::ProcessContext& context, core::ProcessSession& session);
void initialize(const core::Relationship& success, const core::Relationship& failure, const core::Relationship& original, const std::shared_ptr<core::logging::Logger>& logger);
diff --git a/extensions/python/tests/TestExecuteScriptProcessorWithPythonScript.cpp b/extensions/python/tests/TestExecuteScriptProcessorWithPythonScript.cpp
index 34168ba34..f9162a7e5 100644
--- a/extensions/python/tests/TestExecuteScriptProcessorWithPythonScript.cpp
+++ b/extensions/python/tests/TestExecuteScriptProcessorWithPythonScript.cpp
@@ -21,6 +21,7 @@
#include "unit/SingleProcessorTestController.h"
#include "unit/TestBase.h"
+#include "unit/TestUtils.h"
#include "unit/Catch.h"
#include "../../script/ExecuteScript.h"
@@ -30,7 +31,7 @@
namespace org::apache::nifi::minifi::processors::test {
TEST_CASE("Python: hello world") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
REQUIRE(execute_script->setProperty(ExecuteScript::ScriptEngine.name, "python"));
@@ -79,7 +80,7 @@ TEST_CASE("Test both script body and script file set", "[executescriptMisconfigu
}
TEST_CASE("Python: Test session get should return None if there are no flowfiles in the incoming connections") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -97,7 +98,7 @@ def onTrigger(context, session):
}
TEST_CASE("Python: Test Read File", "[executescriptPythonRead]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -126,7 +127,7 @@ def onTrigger(context, session):
}
TEST_CASE("Python: Test Write File", "[executescriptPythonWrite]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -152,7 +153,7 @@ def onTrigger(context, session):
}
TEST_CASE("Python: Test Create", "[executescriptPythonCreate]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -174,7 +175,7 @@ def onTrigger(context, session):
}
TEST_CASE("Python: Test Update Attribute", "[executescriptPythonUpdateAttribute]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -199,7 +200,7 @@ def onTrigger(context, session):
}
TEST_CASE("Python: Test Get Context Property", "[executescriptPythonGetContextProperty]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -220,7 +221,7 @@ def onTrigger(context, session):
TEST_CASE("Python: Test Module Directory property", "[executescriptPythonModuleDirectoryProperty]") {
using org::apache::nifi::minifi::utils::file::get_executable_dir;
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -238,7 +239,7 @@ TEST_CASE("Python: Test Module Directory property", "[executescriptPythonModuleD
}
TEST_CASE("Python: Non existent script file should throw", "[executescriptPythonNonExistentScriptFile]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
@@ -249,7 +250,7 @@ TEST_CASE("Python: Non existent script file should throw", "[executescriptPython
}
TEST_CASE("Python can remove flowfiles", "[ExecuteScript]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<ExecuteScript>();
REQUIRE(execute_script->setProperty(ExecuteScript::ScriptEngine.name, "python"));
@@ -263,7 +264,7 @@ def onTrigger(context, session):
}
TEST_CASE("Python can store states in StateManager", "[ExecuteScript]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<ExecuteScript>("ExecuteScript")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<ExecuteScript>("ExecuteScript")};
const auto execute_script = controller.getProcessor();
LogTestController::getInstance().setTrace<minifi::processors::ExecuteScript>();
REQUIRE(execute_script->setProperty(ExecuteScript::ScriptEngine.name, "python"));
diff --git a/extensions/rocksdb-repos/tests/RepoTests.cpp b/extensions/rocksdb-repos/tests/RepoTests.cpp
index e153cc140..3442ab547 100644
--- a/extensions/rocksdb-repos/tests/RepoTests.cpp
+++ b/extensions/rocksdb-repos/tests/RepoTests.cpp
@@ -319,7 +319,7 @@ TEST_CASE("Test FlowFile Restore", "[TestFFR6]") {
{
std::string data = "banana";
minifi::io::BufferStream content(data);
- std::shared_ptr<core::Processor> processor = std::make_shared<TestProcessor>("dummy");
+ std::shared_ptr<core::Processor> processor = minifi::test::utils::make_processor<TestProcessor>("dummy");
utils::Identifier uuid = processor->getUUID();
REQUIRE(uuid);
inputPtr->setSourceUUID(uuid);
diff --git a/extensions/rocksdb-repos/tests/SwapTests.cpp b/extensions/rocksdb-repos/tests/SwapTests.cpp
index 80ee61678..de7929655 100644
--- a/extensions/rocksdb-repos/tests/SwapTests.cpp
+++ b/extensions/rocksdb-repos/tests/SwapTests.cpp
@@ -22,7 +22,7 @@
#include "unit/TestBase.h"
#include "io/StreamPipe.h"
#include "unit/TestUtils.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/RelationshipDefinition.h"
@@ -99,7 +99,7 @@ TEST_CASE("Connection will on-demand swap flow files") {
ff_repo->loadComponent(content_repo);
ff_repo->start();
- auto processor = std::make_shared<OutputProcessor>("proc");
+ auto processor = minifi::test::utils::make_processor<OutputProcessor>("proc");
auto connection = std::make_shared<minifi::ConnectionImpl>(ff_repo, content_repo, swap_manager, "conn", minifi::utils::IdGenerator::getIdGenerator()->generate());
connection->setSwapThreshold(50);
@@ -114,7 +114,7 @@ TEST_CASE("Connection will on-demand swap flow files") {
processor->triggerAndCommit(context, session_factory);
}
- REQUIRE(connection->getQueueSize() == processor->flow_files_.size());
+ REQUIRE(connection->getQueueSize() == processor->getImpl<OutputProcessor>().flow_files_.size());
minifi::utils::FlowFileQueue& queue = utils::ConnectionTestAccessor::get_queue_(*connection);
// below max threshold live flow files
REQUIRE(utils::FlowFileQueueTestAccessor::get_queue_(queue).size() <= 75);
@@ -129,7 +129,7 @@ TEST_CASE("Connection will on-demand swap flow files") {
});
REQUIRE(got_non_null_flow_file);
REQUIRE(ff->getAttribute("index") == std::to_string(i));
- REQUIRE(ff->getResourceClaim()->getContentFullPath() == processor->flow_files_[i]->getResourceClaim()->getContentFullPath());
+ REQUIRE(ff->getResourceClaim()->getContentFullPath() == processor->getImpl<OutputProcessor>().flow_files_[i]->getResourceClaim()->getContentFullPath());
}
REQUIRE(queue.empty());
diff --git a/extensions/script/ExecuteScript.cpp b/extensions/script/ExecuteScript.cpp
index 88488099f..7ba0890a7 100644
--- a/extensions/script/ExecuteScript.cpp
+++ b/extensions/script/ExecuteScript.cpp
@@ -70,7 +70,7 @@ void ExecuteScript::onSchedule(core::ProcessContext& context, core::ProcessSessi
throw Exception(PROCESS_SCHEDULE_EXCEPTION, "Script File set is not a regular file or does not exist: " + script_file);
}
- script_executor_->initialize(std::move(script_file), std::move(script_body), std::move(module_directory), getMaxConcurrentTasks(), Success, Failure, Original, logger_);
+ script_executor_->initialize(std::move(script_file), std::move(script_body), std::move(module_directory), context.getMaxConcurrentTasks(), Success, Failure, Original, logger_);
}
void ExecuteScript::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
diff --git a/extensions/script/ExecuteScript.h b/extensions/script/ExecuteScript.h
index 3b064155a..3cd8284ec 100644
--- a/extensions/script/ExecuteScript.h
+++ b/extensions/script/ExecuteScript.h
@@ -46,10 +46,7 @@ enum class ScriptEngineOption {
class ExecuteScript : public core::ProcessorImpl {
public:
- explicit ExecuteScript(const std::string_view name, const utils::Identifier &uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ExecuteScript>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Executes a script given the flow file and a process session. "
"The script is responsible for handling the incoming flow file (transfer to SUCCESS or remove, e.g.) as well as "
diff --git a/extensions/sftp/processors/FetchSFTP.cpp b/extensions/sftp/processors/FetchSFTP.cpp
index 6c6ea1a5b..f9d4830dc 100644
--- a/extensions/sftp/processors/FetchSFTP.cpp
+++ b/extensions/sftp/processors/FetchSFTP.cpp
@@ -41,11 +41,6 @@ void FetchSFTP::initialize() {
setSupportedRelationships(Relationships);
}
-FetchSFTP::FetchSFTP(std::string_view name, const utils::Identifier& uuid /*= utils::Identifier()*/)
- : SFTPProcessorBase(name, uuid) {
- logger_ = core::logging::LoggerFactory<FetchSFTP>::getLogger(uuid_);
-}
-
FetchSFTP::~FetchSFTP() = default;
void FetchSFTP::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) {
diff --git a/extensions/sftp/processors/FetchSFTP.h b/extensions/sftp/processors/FetchSFTP.h
index f4228b1d4..7a1d86f8f 100644
--- a/extensions/sftp/processors/FetchSFTP.h
+++ b/extensions/sftp/processors/FetchSFTP.h
@@ -42,7 +42,7 @@ class FetchSFTP : public SFTPProcessorBase {
static constexpr char const *COMPLETION_STRATEGY_MOVE_FILE = "Move File";
static constexpr char const *COMPLETION_STRATEGY_DELETE_FILE = "Delete File";
- explicit FetchSFTP(std::string_view name, const utils::Identifier& uuid = {});
+ using SFTPProcessorBase::SFTPProcessorBase;
~FetchSFTP() override;
EXTENSIONAPI static constexpr const char* Description = "Fetches the content of a file from a remote SFTP server "
diff --git a/extensions/sftp/processors/ListSFTP.cpp b/extensions/sftp/processors/ListSFTP.cpp
index 894250f4b..e1677b327 100644
--- a/extensions/sftp/processors/ListSFTP.cpp
+++ b/extensions/sftp/processors/ListSFTP.cpp
@@ -68,11 +68,6 @@ void ListSFTP::initialize() {
setSupportedRelationships(Relationships);
}
-ListSFTP::ListSFTP(std::string_view name, const utils::Identifier& uuid /*= utils::Identifier()*/)
- : SFTPProcessorBase(name, uuid) {
- logger_ = core::logging::LoggerFactory<ListSFTP>::getLogger(uuid_);
-}
-
ListSFTP::~ListSFTP() = default;
void ListSFTP::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) {
diff --git a/extensions/sftp/processors/ListSFTP.h b/extensions/sftp/processors/ListSFTP.h
index a4fd3a738..daf5824ef 100644
--- a/extensions/sftp/processors/ListSFTP.h
+++ b/extensions/sftp/processors/ListSFTP.h
@@ -55,7 +55,7 @@ class ListSFTP : public SFTPProcessorBase {
static constexpr std::string_view ENTITY_TRACKING_INITIAL_LISTING_TARGET_TRACKING_TIME_WINDOW = "Tracking Time Window";
static constexpr std::string_view ENTITY_TRACKING_INITIAL_LISTING_TARGET_ALL_AVAILABLE = "All Available";
- explicit ListSFTP(std::string_view name, const utils::Identifier& uuid = {});
+ using SFTPProcessorBase::SFTPProcessorBase;
~ListSFTP() override;
EXTENSIONAPI static constexpr const char* Description = "Performs a listing of the files residing on an SFTP server. "
diff --git a/extensions/sftp/processors/PutSFTP.cpp b/extensions/sftp/processors/PutSFTP.cpp
index a2ab3e157..5d0e836c0 100644
--- a/extensions/sftp/processors/PutSFTP.cpp
+++ b/extensions/sftp/processors/PutSFTP.cpp
@@ -42,13 +42,12 @@ void PutSFTP::initialize() {
setSupportedRelationships(Relationships);
}
-PutSFTP::PutSFTP(std::string_view name, const utils::Identifier& uuid /*= utils::Identifier()*/)
- : SFTPProcessorBase(name, uuid),
+PutSFTP::PutSFTP(core::ProcessorMetadata info)
+ : SFTPProcessorBase(info),
create_directory_(false),
batch_size_(0),
reject_zero_byte_(false),
dot_rename_(false) {
- logger_ = core::logging::LoggerFactory<PutSFTP>::getLogger(uuid_);
}
PutSFTP::~PutSFTP() = default;
diff --git a/extensions/sftp/processors/PutSFTP.h b/extensions/sftp/processors/PutSFTP.h
index 87c2ba036..93ecf78c9 100644
--- a/extensions/sftp/processors/PutSFTP.h
+++ b/extensions/sftp/processors/PutSFTP.h
@@ -51,7 +51,7 @@ class PutSFTP : public SFTPProcessorBase {
static constexpr std::string_view CONFLICT_RESOLUTION_FAIL = "FAIL";
static constexpr std::string_view CONFLICT_RESOLUTION_NONE = "NONE";
- explicit PutSFTP(std::string_view name, const utils::Identifier& uuid = {});
+ explicit PutSFTP(core::ProcessorMetadata info);
~PutSFTP() override;
EXTENSIONAPI static constexpr const char* Description = "Sends FlowFiles to an SFTP Server";
diff --git a/extensions/sftp/processors/SFTPProcessorBase.cpp b/extensions/sftp/processors/SFTPProcessorBase.cpp
index 14721a2bc..c9d620233 100644
--- a/extensions/sftp/processors/SFTPProcessorBase.cpp
+++ b/extensions/sftp/processors/SFTPProcessorBase.cpp
@@ -43,8 +43,8 @@
namespace org::apache::nifi::minifi::processors {
-SFTPProcessorBase::SFTPProcessorBase(const std::string_view name, const utils::Identifier& uuid)
- : ProcessorImpl(name, uuid),
+SFTPProcessorBase::SFTPProcessorBase(core::ProcessorMetadata info)
+ : ProcessorImpl(info),
connection_timeout_(0),
data_timeout_(0),
strict_host_checking_(false),
diff --git a/extensions/sftp/processors/SFTPProcessorBase.h b/extensions/sftp/processors/SFTPProcessorBase.h
index f1135f731..628fd652e 100644
--- a/extensions/sftp/processors/SFTPProcessorBase.h
+++ b/extensions/sftp/processors/SFTPProcessorBase.h
@@ -27,7 +27,7 @@
#include <vector>
#include "FlowFileRecord.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -40,7 +40,7 @@ namespace org::apache::nifi::minifi::processors {
class SFTPProcessorBase : public core::ProcessorImpl {
public:
- SFTPProcessorBase(std::string_view name, const utils::Identifier& uuid);
+ SFTPProcessorBase(core::ProcessorMetadata info);
~SFTPProcessorBase() override;
static constexpr std::string_view PROXY_TYPE_DIRECT = "DIRECT";
@@ -158,8 +158,6 @@ class SFTPProcessorBase : public core::ProcessorImpl {
void notifyStop() override;
protected:
- std::shared_ptr<core::logging::Logger> logger_;
-
std::chrono::milliseconds connection_timeout_;
std::chrono::milliseconds data_timeout_;
std::string host_key_file_;
diff --git a/extensions/splunk/PutSplunkHTTP.cpp b/extensions/splunk/PutSplunkHTTP.cpp
index a70d1c237..30280335f 100644
--- a/extensions/splunk/PutSplunkHTTP.cpp
+++ b/extensions/splunk/PutSplunkHTTP.cpp
@@ -120,7 +120,7 @@ void PutSplunkHTTP::onSchedule(core::ProcessContext& context, core::ProcessSessi
return client;
};
- client_queue_ = utils::ResourceQueue<http::HTTPClient>::create(create_client, getMaxConcurrentTasks(), std::nullopt, logger_);
+ client_queue_ = utils::ResourceQueue<http::HTTPClient>::create(create_client, context.getMaxConcurrentTasks(), std::nullopt, logger_);
source_type_ = utils::parseOptionalProperty(context, PutSplunkHTTP::SourceType);
source_ = utils::parseOptionalProperty(context, PutSplunkHTTP::Source);
host_ = utils::parseOptionalProperty(context, PutSplunkHTTP::Host);
diff --git a/extensions/splunk/PutSplunkHTTP.h b/extensions/splunk/PutSplunkHTTP.h
index 1dcc4f392..eb694dcb5 100644
--- a/extensions/splunk/PutSplunkHTTP.h
+++ b/extensions/splunk/PutSplunkHTTP.h
@@ -34,10 +34,7 @@ namespace org::apache::nifi::minifi::extensions::splunk {
class PutSplunkHTTP final : public SplunkHECProcessor {
public:
- explicit PutSplunkHTTP(std::string_view name, const utils::Identifier& uuid = {})
- : SplunkHECProcessor(name, uuid) {
- logger_ = core::logging::LoggerFactory<PutSplunkHTTP>::getLogger(uuid_);
- }
+ using SplunkHECProcessor::SplunkHECProcessor;
PutSplunkHTTP(const PutSplunkHTTP&) = delete;
PutSplunkHTTP(PutSplunkHTTP&&) = delete;
PutSplunkHTTP& operator=(const PutSplunkHTTP&) = delete;
diff --git a/extensions/splunk/QuerySplunkIndexingStatus.h b/extensions/splunk/QuerySplunkIndexingStatus.h
index 4f4014ae6..1973735da 100644
--- a/extensions/splunk/QuerySplunkIndexingStatus.h
+++ b/extensions/splunk/QuerySplunkIndexingStatus.h
@@ -35,9 +35,7 @@ namespace org::apache::nifi::minifi::extensions::splunk {
class QuerySplunkIndexingStatus final : public SplunkHECProcessor {
public:
- explicit QuerySplunkIndexingStatus(std::string_view name, const utils::Identifier& uuid = {})
- : SplunkHECProcessor(name, uuid) {
- }
+ using SplunkHECProcessor::SplunkHECProcessor;
QuerySplunkIndexingStatus(const QuerySplunkIndexingStatus&) = delete;
QuerySplunkIndexingStatus(QuerySplunkIndexingStatus&&) = delete;
QuerySplunkIndexingStatus& operator=(const QuerySplunkIndexingStatus&) = delete;
diff --git a/extensions/splunk/SplunkHECProcessor.h b/extensions/splunk/SplunkHECProcessor.h
index 6fc8641b6..766f2ee3b 100644
--- a/extensions/splunk/SplunkHECProcessor.h
+++ b/extensions/splunk/SplunkHECProcessor.h
@@ -22,7 +22,7 @@
#include "controllers/SSLContextServiceInterface.h"
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "http/HTTPClient.h"
@@ -69,10 +69,7 @@ class SplunkHECProcessor : public core::ProcessorImpl {
SSLContext
});
-
- explicit SplunkHECProcessor(std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- }
+ using ProcessorImpl::ProcessorImpl;
~SplunkHECProcessor() override = default;
void initialize() override;
diff --git a/extensions/splunk/tests/PutSplunkHTTPTests.cpp b/extensions/splunk/tests/PutSplunkHTTPTests.cpp
index 93712a34e..07da47ddc 100644
--- a/extensions/splunk/tests/PutSplunkHTTPTests.cpp
+++ b/extensions/splunk/tests/PutSplunkHTTPTests.cpp
@@ -36,10 +36,10 @@ TEST_CASE("PutSplunkHTTP tests", "[putsplunkhttp]") {
TestController test_controller;
auto plan = test_controller.createPlan();
- auto write_to_flow_file = dynamic_cast<WriteToFlowFileTestProcessor*>(plan->addProcessor("WriteToFlowFileTestProcessor", "write_to_flow_file"));
- auto put_splunk_http = dynamic_cast<PutSplunkHTTP*>(plan->addProcessor("PutSplunkHTTP", "put_splunk_http"));
- auto read_from_success = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_success"));
- auto read_from_failure = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_failure"));
+ auto write_to_flow_file = plan->addProcessor<WriteToFlowFileTestProcessor>("write_to_flow_file");
+ auto put_splunk_http = plan->addProcessor("PutSplunkHTTP", "put_splunk_http");
+ auto read_from_success = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_success");
+ auto read_from_failure = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_failure");
plan->addConnection(write_to_flow_file, WriteToFlowFileTestProcessor::Success, put_splunk_http);
plan->addConnection(put_splunk_http, PutSplunkHTTP::Success, read_from_success);
@@ -53,20 +53,20 @@ TEST_CASE("PutSplunkHTTP tests", "[putsplunkhttp]") {
plan->setProperty(put_splunk_http, PutSplunkHTTP::Token, MockSplunkHEC::TOKEN);
plan->setProperty(put_splunk_http, PutSplunkHTTP::SplunkRequestChannel, "a12254b4-f481-435d-896d-3b6033eabe58");
- write_to_flow_file->setContent("foobar");
+ write_to_flow_file.get().setContent("foobar");
SECTION("Happy path") {
mock_splunk_hec.setAssertions([](const struct mg_request_info *request_info) {
CHECK(request_info->query_string == nullptr);
});
test_controller.runSession(plan);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 0);
- CHECK(read_from_success->numberOfFlowFilesRead() == 1);
- CHECK(read_from_success->readFlowFileWithContent("foobar"));
- CHECK(read_from_success->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_STATUS_CODE, "200"));
- CHECK(read_from_success->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_CODE, "0"));
- CHECK(read_from_success->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_TIME));
- CHECK(read_from_success->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_ACK_ID));
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success.get().numberOfFlowFilesRead() == 1);
+ CHECK(read_from_success.get().readFlowFileWithContent("foobar"));
+ CHECK(read_from_success.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_STATUS_CODE, "200"));
+ CHECK(read_from_success.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_CODE, "0"));
+ CHECK(read_from_success.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_TIME));
+ CHECK(read_from_success.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_ACK_ID));
}
SECTION("Happy path with query arguments") {
@@ -83,26 +83,26 @@ TEST_CASE("PutSplunkHTTP tests", "[putsplunkhttp]") {
CHECK(query_string.find("index=qux") != std::string::npos);
});
test_controller.runSession(plan);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 0);
- CHECK(read_from_success->numberOfFlowFilesRead() == 1);
- CHECK(read_from_success->readFlowFileWithContent("foobar"));
- CHECK(read_from_success->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_STATUS_CODE, "200"));
- CHECK(read_from_success->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_CODE, "0"));
- CHECK(read_from_success->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_TIME));
- CHECK(read_from_success->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_ACK_ID));
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success.get().numberOfFlowFilesRead() == 1);
+ CHECK(read_from_success.get().readFlowFileWithContent("foobar"));
+ CHECK(read_from_success.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_STATUS_CODE, "200"));
+ CHECK(read_from_success.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_CODE, "0"));
+ CHECK(read_from_success.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_TIME));
+ CHECK(read_from_success.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_ACK_ID));
}
SECTION("Invalid Token") {
constexpr const char* invalid_token = "Splunk 00000000-0000-0000-0000-000000000000";
plan->setProperty(put_splunk_http, PutSplunkHTTP::Token, invalid_token);
test_controller.runSession(plan);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 1);
- CHECK(read_from_success->numberOfFlowFilesRead() == 0);
- CHECK(read_from_failure->readFlowFileWithContent("foobar"));
- CHECK(read_from_failure->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_STATUS_CODE, "403"));
- CHECK(read_from_failure->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_CODE, "4"));
- CHECK(read_from_failure->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_TIME));
- CHECK_FALSE(read_from_failure->readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_ACK_ID));
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 1);
+ CHECK(read_from_success.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure.get().readFlowFileWithContent("foobar"));
+ CHECK(read_from_failure.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_STATUS_CODE, "403"));
+ CHECK(read_from_failure.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_CODE, "4"));
+ CHECK(read_from_failure.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_TIME));
+ CHECK_FALSE(read_from_failure.get().readFlowFileWithAttribute(org::apache::nifi::minifi::extensions::splunk::SPLUNK_ACK_ID));
}
}
@@ -128,9 +128,9 @@ TEST_CASE("PutSplunkHTTP content type tests", "[putsplunkhttpcontenttype]") {
TestController test_controller;
auto plan = test_controller.createPlan();
- auto write_to_flow_file = dynamic_cast<WriteToFlowFileTestProcessor*>(plan->addProcessor("WriteToFlowFileTestProcessor", "write_to_flow_file"));
- auto update_attribute = dynamic_cast<UpdateAttribute*>(plan->addProcessor("UpdateAttribute", "update_attribute"));
- auto put_splunk_http = dynamic_cast<PutSplunkHTTP*>(plan->addProcessor("PutSplunkHTTP", "put_splunk_http"));
+ auto write_to_flow_file = plan->addProcessor<WriteToFlowFileTestProcessor>("write_to_flow_file");
+ auto update_attribute = plan->addProcessor("UpdateAttribute", "update_attribute");
+ auto put_splunk_http = plan->addProcessor("PutSplunkHTTP", "put_splunk_http");
plan->addConnection(write_to_flow_file, WriteToFlowFileTestProcessor::Success, update_attribute);
plan->addConnection(update_attribute, UpdateAttribute::Success, put_splunk_http);
@@ -141,7 +141,7 @@ TEST_CASE("PutSplunkHTTP content type tests", "[putsplunkhttpcontenttype]") {
plan->setProperty(put_splunk_http, PutSplunkHTTP::Token, MockSplunkHEC::TOKEN);
plan->setProperty(put_splunk_http, PutSplunkHTTP::SplunkRequestChannel, "a12254b4-f481-435d-896d-3b6033eabe58");
- write_to_flow_file->setContent("foobar");
+ write_to_flow_file.get().setContent("foobar");
SECTION("Content Type without Property or Attribute") {
mock_splunk_hec.setAssertions(ContentTypeValidator("application/x-www-form-urlencoded"));
diff --git a/extensions/splunk/tests/QuerySplunkIndexingStatusTests.cpp b/extensions/splunk/tests/QuerySplunkIndexingStatusTests.cpp
index 9100162e4..8cc4a6fb3 100644
--- a/extensions/splunk/tests/QuerySplunkIndexingStatusTests.cpp
+++ b/extensions/splunk/tests/QuerySplunkIndexingStatusTests.cpp
@@ -37,13 +37,13 @@ TEST_CASE("QuerySplunkIndexingStatus tests", "[querysplunkindexingstatus]") {
TestController test_controller;
auto plan = test_controller.createPlan();
- auto write_to_flow_file = dynamic_cast<WriteToFlowFileTestProcessor*>(plan->addProcessor("WriteToFlowFileTestProcessor", "write_to_flow_file"));
- auto update_attribute = dynamic_cast<UpdateAttribute*>(plan->addProcessor("UpdateAttribute", "update_attribute"));
- auto query_splunk_indexing_status = dynamic_cast<QuerySplunkIndexingStatus*>(plan->addProcessor("QuerySplunkIndexingStatus", "query_splunk_indexing_status"));
- auto read_from_acknowledged = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_acknowledged"));
- auto read_from_undetermined = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_undetermined"));
- auto read_from_unacknowledged = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_unacknowledged"));
- auto read_from_failure = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_failure"));
+ auto write_to_flow_file = plan->addProcessor<WriteToFlowFileTestProcessor>("write_to_flow_file");
+ auto update_attribute = plan->addProcessor("UpdateAttribute", "update_attribute");
+ auto query_splunk_indexing_status = plan->addProcessor("QuerySplunkIndexingStatus", "query_splunk_indexing_status");
+ auto read_from_acknowledged = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_acknowledged");
+ auto read_from_undetermined = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_undetermined");
+ auto read_from_unacknowledged = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_unacknowledged");
+ auto read_from_failure = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_failure");
plan->addConnection(write_to_flow_file, WriteToFlowFileTestProcessor::Success, update_attribute);
plan->addConnection(update_attribute, UpdateAttribute ::Success, query_splunk_indexing_status);
@@ -65,35 +65,35 @@ TEST_CASE("QuerySplunkIndexingStatus tests", "[querysplunkindexingstatus]") {
auto response_timestamp = std::to_string(std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count());
plan->setDynamicProperty(update_attribute, org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_TIME, response_timestamp);
- write_to_flow_file->setContent("foobar");
+ write_to_flow_file.get().setContent("foobar");
SECTION("Querying indexed id") {
plan->setDynamicProperty(update_attribute, org::apache::nifi::minifi::extensions::splunk::SPLUNK_ACK_ID, std::to_string(MockSplunkHEC::indexed_events[0]));
test_controller.runSession(plan);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 0);
- CHECK(read_from_undetermined->numberOfFlowFilesRead() == 0);
- CHECK(read_from_unacknowledged->numberOfFlowFilesRead() == 0);
- CHECK(read_from_acknowledged->numberOfFlowFilesRead() == 1);
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_undetermined.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_unacknowledged.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_acknowledged.get().numberOfFlowFilesRead() == 1);
}
SECTION("Querying not indexed id") {
plan->setDynamicProperty(update_attribute, org::apache::nifi::minifi::extensions::splunk::SPLUNK_ACK_ID, "100");
query_splunk_indexing_status->setPenalizationPeriod(50ms);
test_controller.runSession(plan);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 0);
- CHECK(read_from_undetermined->numberOfFlowFilesRead() == 0); // result penalized
- CHECK(read_from_unacknowledged->numberOfFlowFilesRead() == 0);
- CHECK(read_from_acknowledged->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_undetermined.get().numberOfFlowFilesRead() == 0); // result penalized
+ CHECK(read_from_unacknowledged.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_acknowledged.get().numberOfFlowFilesRead() == 0);
- write_to_flow_file->setContent("");
+ write_to_flow_file.get().setContent("");
plan->reset();
std::this_thread::sleep_for(100ms);
test_controller.runSession(plan);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 0);
- CHECK(read_from_undetermined->numberOfFlowFilesRead() == 1);
- CHECK(read_from_unacknowledged->numberOfFlowFilesRead() == 0);
- CHECK(read_from_acknowledged->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_undetermined.get().numberOfFlowFilesRead() == 1);
+ CHECK(read_from_unacknowledged.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_acknowledged.get().numberOfFlowFilesRead() == 0);
}
SECTION("Querying not indexed old id") {
@@ -102,10 +102,10 @@ TEST_CASE("QuerySplunkIndexingStatus tests", "[querysplunkindexingstatus]") {
plan->setDynamicProperty(update_attribute, org::apache::nifi::minifi::extensions::splunk::SPLUNK_RESPONSE_TIME, response_timestamp);
test_controller.runSession(plan);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 0);
- CHECK(read_from_undetermined->numberOfFlowFilesRead() == 0);
- CHECK(read_from_unacknowledged->numberOfFlowFilesRead() == 1);
- CHECK(read_from_acknowledged->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_undetermined.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_unacknowledged.get().numberOfFlowFilesRead() == 1);
+ CHECK(read_from_acknowledged.get().numberOfFlowFilesRead() == 0);
}
SECTION("Multiple inputs with same id") {
@@ -120,10 +120,10 @@ TEST_CASE("QuerySplunkIndexingStatus tests", "[querysplunkindexingstatus]") {
plan->runProcessor(read_from_undetermined);
plan->runProcessor(read_from_unacknowledged);
plan->runProcessor(read_from_acknowledged);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 4);
- CHECK(read_from_undetermined->numberOfFlowFilesRead() == 0);
- CHECK(read_from_unacknowledged->numberOfFlowFilesRead() == 0);
- CHECK(read_from_acknowledged->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 4);
+ CHECK(read_from_undetermined.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_unacknowledged.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_acknowledged.get().numberOfFlowFilesRead() == 0);
}
SECTION("MaxQuerySize can limit the number of queries") {
@@ -138,10 +138,10 @@ TEST_CASE("QuerySplunkIndexingStatus tests", "[querysplunkindexingstatus]") {
SECTION("Input flow file has no attributes") {
test_controller.runSession(plan);
- CHECK(read_from_failure->numberOfFlowFilesRead() == 1);
- CHECK(read_from_undetermined->numberOfFlowFilesRead() == 0);
- CHECK(read_from_unacknowledged->numberOfFlowFilesRead() == 0);
- CHECK(read_from_acknowledged->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure.get().numberOfFlowFilesRead() == 1);
+ CHECK(read_from_undetermined.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_unacknowledged.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_acknowledged.get().numberOfFlowFilesRead() == 0);
}
SECTION("Invalid index") {
diff --git a/extensions/sql/processors/ExecuteSQL.cpp b/extensions/sql/processors/ExecuteSQL.cpp
index 55543ca54..f39151758 100644
--- a/extensions/sql/processors/ExecuteSQL.cpp
+++ b/extensions/sql/processors/ExecuteSQL.cpp
@@ -32,10 +32,6 @@ namespace org::apache::nifi::minifi::processors {
const std::string ExecuteSQL::RESULT_ROW_COUNT = "executesql.row.count";
const std::string ExecuteSQL::INPUT_FLOW_FILE_UUID = "input.flowfile.uuid";
-ExecuteSQL::ExecuteSQL(std::string_view name, const utils::Identifier& uuid)
- : SQLProcessor(name, uuid, core::logging::LoggerFactory<ExecuteSQL>::getLogger(uuid)) {
-}
-
void ExecuteSQL::initialize() {
setSupportedProperties(Properties);
setSupportedRelationships(Relationships);
diff --git a/extensions/sql/processors/ExecuteSQL.h b/extensions/sql/processors/ExecuteSQL.h
index 8a6e9ccce..716fb9a35 100644
--- a/extensions/sql/processors/ExecuteSQL.h
+++ b/extensions/sql/processors/ExecuteSQL.h
@@ -35,7 +35,7 @@ namespace org::apache::nifi::minifi::processors {
class ExecuteSQL : public SQLProcessor, public FlowFileSource {
public:
- explicit ExecuteSQL(std::string_view name, const utils::Identifier& uuid = {});
+ using SQLProcessor::SQLProcessor;
EXTENSIONAPI static constexpr const char* Description = "Execute provided SQL query. "
"Query result rows will be outputted as new flow files with attribute keys equal to result column names and values equal to result values. "
diff --git a/extensions/sql/processors/PutSQL.cpp b/extensions/sql/processors/PutSQL.cpp
index 888330810..63f0b41ea 100644
--- a/extensions/sql/processors/PutSQL.cpp
+++ b/extensions/sql/processors/PutSQL.cpp
@@ -29,10 +29,6 @@
namespace org::apache::nifi::minifi::processors {
-PutSQL::PutSQL(std::string_view name, const utils::Identifier& uuid)
- : SQLProcessor(name, uuid, core::logging::LoggerFactory<PutSQL>::getLogger(uuid)) {
-}
-
void PutSQL::initialize() {
setSupportedProperties(Properties);
setSupportedRelationships(Relationships);
diff --git a/extensions/sql/processors/PutSQL.h b/extensions/sql/processors/PutSQL.h
index 8acb06863..0c4550747 100644
--- a/extensions/sql/processors/PutSQL.h
+++ b/extensions/sql/processors/PutSQL.h
@@ -33,7 +33,7 @@ namespace org::apache::nifi::minifi::processors {
class PutSQL : public SQLProcessor {
public:
- explicit PutSQL(std::string_view name, const utils::Identifier& uuid = {});
+ using SQLProcessor::SQLProcessor;
EXTENSIONAPI static constexpr const char* Description =
"Executes a SQL UPDATE or INSERT command. The content of an incoming FlowFile is expected to be the SQL command to execute. "
diff --git a/extensions/sql/processors/QueryDatabaseTable.cpp b/extensions/sql/processors/QueryDatabaseTable.cpp
index c5507ca61..6c5f66b3d 100644
--- a/extensions/sql/processors/QueryDatabaseTable.cpp
+++ b/extensions/sql/processors/QueryDatabaseTable.cpp
@@ -40,10 +40,6 @@ const std::string QueryDatabaseTable::RESULT_ROW_COUNT = "querydbtable.row.count
const std::string QueryDatabaseTable::TABLENAME_KEY = "tablename";
const std::string QueryDatabaseTable::MAXVALUE_KEY_PREFIX = "maxvalue.";
-QueryDatabaseTable::QueryDatabaseTable(std::string_view name, const utils::Identifier& uuid)
- : SQLProcessor(name, uuid, core::logging::LoggerFactory<QueryDatabaseTable>::getLogger(uuid)) {
-}
-
void QueryDatabaseTable::initialize() {
setSupportedProperties(Properties);
setSupportedRelationships(Relationships);
diff --git a/extensions/sql/processors/QueryDatabaseTable.h b/extensions/sql/processors/QueryDatabaseTable.h
index 740328b30..930d9b51f 100644
--- a/extensions/sql/processors/QueryDatabaseTable.h
+++ b/extensions/sql/processors/QueryDatabaseTable.h
@@ -40,7 +40,7 @@ namespace org::apache::nifi::minifi::processors {
class QueryDatabaseTable: public SQLProcessor, public FlowFileSource {
public:
- explicit QueryDatabaseTable(std::string_view name, const utils::Identifier& uuid = {});
+ using SQLProcessor::SQLProcessor;
EXTENSIONAPI static const std::string RESULT_TABLE_NAME;
EXTENSIONAPI static const std::string RESULT_ROW_COUNT;
diff --git a/extensions/sql/processors/SQLProcessor.h b/extensions/sql/processors/SQLProcessor.h
index ea326a99a..2cd34194d 100644
--- a/extensions/sql/processors/SQLProcessor.h
+++ b/extensions/sql/processors/SQLProcessor.h
@@ -21,7 +21,7 @@
#pragma once
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -41,9 +41,7 @@ class SQLProcessor: public core::ProcessorImpl {
EXTENSIONAPI static constexpr auto Properties = std::to_array<core::PropertyReference>({DBControllerService});
protected:
- SQLProcessor(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger)
- : core::ProcessorImpl(name, uuid), logger_(std::move(logger)) {
- }
+ using ProcessorImpl::ProcessorImpl;
static std::vector<std::string> collectArguments(const std::shared_ptr<core::FlowFile>& flow_file);
@@ -57,7 +55,6 @@ class SQLProcessor: public core::ProcessorImpl {
connection_.reset();
}
- std::shared_ptr<core::logging::Logger> logger_;
std::shared_ptr<sql::controllers::DatabaseService> db_service_;
std::unique_ptr<sql::Connection> connection_;
};
diff --git a/extensions/sql/tests/PutSQLTests.cpp b/extensions/sql/tests/PutSQLTests.cpp
index 5c83d0ff4..6d15ba546 100644
--- a/extensions/sql/tests/PutSQLTests.cpp
+++ b/extensions/sql/tests/PutSQLTests.cpp
@@ -16,6 +16,7 @@
* limitations under the License.
*/
#include "unit/TestBase.h"
+#include "unit/TestUtils.h"
#include "unit/Catch.h"
#include "SQLTestController.h"
@@ -28,7 +29,7 @@
TEST_CASE("Test Creation of PutSQL", "[PutSQLCreate]") {
TestController testController;
std::shared_ptr<core::Processor>
- processor = std::make_shared<org::apache::nifi::minifi::processors::PutSQL>("processorname");
+ processor = minifi::test::utils::make_processor<org::apache::nifi::minifi::processors::PutSQL>("processorname");
REQUIRE(processor->getName() == "processorname");
}
diff --git a/extensions/standard-processors/modbus/FetchModbusTcp.cpp b/extensions/standard-processors/modbus/FetchModbusTcp.cpp
index 8e39db378..dccdad866 100644
--- a/extensions/standard-processors/modbus/FetchModbusTcp.cpp
+++ b/extensions/standard-processors/modbus/FetchModbusTcp.cpp
@@ -41,10 +41,10 @@ void FetchModbusTcp::onSchedule(core::ProcessContext& context, core::ProcessSess
}
// if the required properties are missing or empty even before evaluating the EL expression, then we can throw in onSchedule, before we waste any flow files
- if (context.getProperty(Hostname).value_or(std::string{}).empty()) {
+ if (!context.getProperty(Hostname.name)) {
throw Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "missing hostname"};
}
- if (context.getProperty(Port).value_or(std::string{}).empty()) {
+ if (!context.hasNonEmptyProperty(Port.name)) {
throw Exception{ExceptionType::PROCESS_SCHEDULE_EXCEPTION, "missing port"};
}
@@ -73,7 +73,7 @@ void FetchModbusTcp::onSchedule(core::ProcessContext& context, core::ProcessSess
}
void FetchModbusTcp::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
- const auto flow_file = getOrCreateFlowFile(session);
+ const auto flow_file = getOrCreateFlowFile(context, session);
if (!flow_file) {
logger_->log_error("No flowfile to work on");
return;
@@ -117,8 +117,8 @@ void FetchModbusTcp::initialize() {
setSupportedRelationships(Relationships);
}
-std::shared_ptr<core::FlowFile> FetchModbusTcp::getOrCreateFlowFile(core::ProcessSession& session) const {
- if (hasIncomingConnections()) {
+std::shared_ptr<core::FlowFile> FetchModbusTcp::getOrCreateFlowFile(core::ProcessContext& context, core::ProcessSession& session) const {
+ if (context.hasIncomingConnections()) {
return session.get();
}
return session.create();
diff --git a/extensions/standard-processors/modbus/FetchModbusTcp.h b/extensions/standard-processors/modbus/FetchModbusTcp.h
index 8daafde1c..3b21b9da0 100644
--- a/extensions/standard-processors/modbus/FetchModbusTcp.h
+++ b/extensions/standard-processors/modbus/FetchModbusTcp.h
@@ -31,10 +31,7 @@ class ReadModbusFunction;
class FetchModbusTcp final : public core::ProcessorImpl {
public:
- explicit FetchModbusTcp(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<FetchModbusTcp>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr auto Description = "Processor able to read data from industrial PLCs using Modbus TCP/IP";
@@ -127,7 +124,7 @@ class FetchModbusTcp final : public core::ProcessorImpl {
asio::awaitable<nonstd::expected<core::RecordField, std::error_code>> sendRequestAndReadResponse(utils::net::ConnectionHandlerBase& connection_handler,
const ReadModbusFunction& read_modbus_function);
std::unordered_map<std::string, std::unique_ptr<ReadModbusFunction>> getAddressMap(core::ProcessContext& context, const core::FlowFile& flow_file);
- std::shared_ptr<core::FlowFile> getOrCreateFlowFile(core::ProcessSession& session) const;
+ std::shared_ptr<core::FlowFile> getOrCreateFlowFile(core::ProcessContext& context, core::ProcessSession& session) const;
void removeExpiredConnections();
asio::io_context io_context_;
diff --git a/extensions/standard-processors/processors/AppendHostInfo.h b/extensions/standard-processors/processors/AppendHostInfo.h
index 55bd7fc4e..f55d6e839 100644
--- a/extensions/standard-processors/processors/AppendHostInfo.h
+++ b/extensions/standard-processors/processors/AppendHostInfo.h
@@ -27,7 +27,7 @@
#include <utility>
#include "core/Property.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/ProcessContext.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -42,10 +42,7 @@ class AppendHostInfo : public core::ProcessorImpl {
static constexpr const char* REFRESH_POLICY_ON_TRIGGER = "On every trigger";
static constexpr const char* REFRESH_POLICY_ON_SCHEDULE = "On schedule";
- explicit AppendHostInfo(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<AppendHostInfo>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~AppendHostInfo() override = default;
EXTENSIONAPI static constexpr const char* Description = "Appends host information such as IP address and hostname as an attribute to incoming flowfiles.";
diff --git a/extensions/standard-processors/processors/AttributesToJSON.h b/extensions/standard-processors/processors/AttributesToJSON.h
index 2e0c421ec..4e8fbeab6 100644
--- a/extensions/standard-processors/processors/AttributesToJSON.h
+++ b/extensions/standard-processors/processors/AttributesToJSON.h
@@ -30,7 +30,7 @@
#include "rapidjson/document.h"
#include "core/FlowFile.h"
#include "core/logging/LoggerFactory.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -118,10 +118,7 @@ class AttributesToJSON : public core::ProcessorImpl {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit AttributesToJSON(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<AttributesToJSON>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
void initialize() override;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
diff --git a/extensions/standard-processors/processors/DefragmentText.h b/extensions/standard-processors/processors/DefragmentText.h
index 518134afd..329d64d4d 100644
--- a/extensions/standard-processors/processors/DefragmentText.h
+++ b/extensions/standard-processors/processors/DefragmentText.h
@@ -59,10 +59,7 @@ namespace org::apache::nifi::minifi::processors {
class DefragmentText : public core::ProcessorImpl {
public:
- explicit DefragmentText(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<DefragmentText>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "DefragmentText splits and merges incoming flowfiles so cohesive messages are not split between them. "
"It can handle multiple inputs differentiated by the absolute.path flow file attribute.";
diff --git a/extensions/standard-processors/processors/ExtractText.h b/extensions/standard-processors/processors/ExtractText.h
index dd54485a6..2c878502f 100644
--- a/extensions/standard-processors/processors/ExtractText.h
+++ b/extensions/standard-processors/processors/ExtractText.h
@@ -25,7 +25,7 @@
#include <utility>
#include <vector>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/ProcessContext.h"
#include "core/PropertyDefinitionBuilder.h"
diff --git a/extensions/standard-processors/processors/FetchFile.h b/extensions/standard-processors/processors/FetchFile.h
index 896ad30e5..7ec2e495e 100644
--- a/extensions/standard-processors/processors/FetchFile.h
+++ b/extensions/standard-processors/processors/FetchFile.h
@@ -20,7 +20,7 @@
#include <string>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -84,10 +84,7 @@ namespace org::apache::nifi::minifi::processors {
class FetchFile final : public core::ProcessorImpl {
public:
- explicit FetchFile(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<FetchFile>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Reads the contents of a file from disk and streams it into the contents of an incoming FlowFile. "
"Once this is done, the file is optionally moved elsewhere or deleted to help keep the file system organized.";
diff --git a/extensions/standard-processors/processors/GenerateFlowFile.cpp b/extensions/standard-processors/processors/GenerateFlowFile.cpp
index 24e27170e..7ff2ee3ac 100644
--- a/extensions/standard-processors/processors/GenerateFlowFile.cpp
+++ b/extensions/standard-processors/processors/GenerateFlowFile.cpp
@@ -84,8 +84,7 @@ void GenerateFlowFile::onSchedule(core::ProcessContext& context, core::ProcessSe
| utils::valueOrElse([]() {return false;});
bool is_unique = utils::parseOptionalBoolProperty(context, UniqueFlowFiles).value_or(true);
- const auto custom_text_without_evaluation = context.getProperty(CustomText.name);
- const bool has_custom_text = custom_text_without_evaluation.has_value() && !custom_text_without_evaluation->empty();
+ const bool has_custom_text = context.hasNonEmptyProperty(CustomText.name);
file_size_ = utils::parseDataSizeProperty(context, FileSize);
batch_size_ = utils::parseU64Property(context, BatchSize);
diff --git a/extensions/standard-processors/processors/GenerateFlowFile.h b/extensions/standard-processors/processors/GenerateFlowFile.h
index a3fb0a698..83b6d299c 100644
--- a/extensions/standard-processors/processors/GenerateFlowFile.h
+++ b/extensions/standard-processors/processors/GenerateFlowFile.h
@@ -24,7 +24,7 @@
#include <utility>
#include <vector>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/ProcessContext.h"
#include "core/PropertyDefinition.h"
diff --git a/extensions/standard-processors/processors/GetFile.cpp b/extensions/standard-processors/processors/GetFile.cpp
index ec9cbfc1c..53a8d4d6f 100644
--- a/extensions/standard-processors/processors/GetFile.cpp
+++ b/extensions/standard-processors/processors/GetFile.cpp
@@ -68,7 +68,7 @@ void GetFile::onSchedule(core::ProcessContext& context, core::ProcessSessionFact
}
}
-void GetFile::onTrigger(core::ProcessContext&, core::ProcessSession& session) {
+void GetFile::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
const bool is_dir_empty_before_poll = isListingEmpty();
logger_->log_debug("Listing is {} before polling directory", is_dir_empty_before_poll ? "empty" : "not empty");
if (is_dir_empty_before_poll) {
@@ -81,7 +81,7 @@ void GetFile::onTrigger(core::ProcessContext&, core::ProcessSession& session) {
const bool is_dir_empty_after_poll = isListingEmpty();
logger_->log_debug("Listing is {} after polling directory", is_dir_empty_after_poll ? "empty" : "not empty");
if (is_dir_empty_after_poll) {
- yield();
+ context.yield();
return;
}
@@ -191,7 +191,7 @@ void GetFile::performListing(const GetFileRequest &request) {
if (fileMatchesRequestCriteria(fullpath, filename, request)) {
putListing(fullpath);
}
- return isRunning();
+ return true;
};
utils::file::list_dir(request.inputDirectory, callback, logger_, request.recursive);
}
diff --git a/extensions/standard-processors/processors/GetFile.h b/extensions/standard-processors/processors/GetFile.h
index 1e127b8e1..09fc01133 100644
--- a/extensions/standard-processors/processors/GetFile.h
+++ b/extensions/standard-processors/processors/GetFile.h
@@ -24,7 +24,7 @@
#include <atomic>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
@@ -54,7 +54,7 @@ struct GetFileRequest {
class GetFileMetrics : public core::ProcessorMetricsImpl {
public:
- explicit GetFileMetrics(const core::ProcessorApi& source_processor)
+ explicit GetFileMetrics(const core::ProcessorImpl& source_processor)
: core::ProcessorMetricsImpl(source_processor) {
}
@@ -84,10 +84,9 @@ class GetFileMetrics : public core::ProcessorMetricsImpl {
class GetFile : public core::ProcessorImpl {
public:
- explicit GetFile(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
+ explicit GetFile(core::ProcessorMetadata info)
+ : ProcessorImpl(info) {
metrics_ = gsl::make_not_null(std::make_shared<GetFileMetrics>(*this));
- logger_ = core::logging::LoggerFactory<GetFile>::getLogger(uuid_);
}
~GetFile() override = default;
diff --git a/extensions/standard-processors/processors/GetTCP.h b/extensions/standard-processors/processors/GetTCP.h
index 9562d0c03..9c8b16363 100644
--- a/extensions/standard-processors/processors/GetTCP.h
+++ b/extensions/standard-processors/processors/GetTCP.h
@@ -27,7 +27,7 @@
#include "utils/Literals.h"
#include "minifi-cpp/core/OutputAttributeDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -46,10 +46,7 @@ namespace org::apache::nifi::minifi::processors {
class GetTCP : public core::ProcessorImpl {
public:
- explicit GetTCP(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<GetTCP>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~GetTCP() override {
if (client_) {
diff --git a/extensions/standard-processors/processors/HashContent.h b/extensions/standard-processors/processors/HashContent.h
index 3d4507d84..fadb2af09 100644
--- a/extensions/standard-processors/processors/HashContent.h
+++ b/extensions/standard-processors/processors/HashContent.h
@@ -32,7 +32,7 @@
#include <sstream>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/RelationshipDefinition.h"
@@ -141,10 +141,7 @@ static const std::map<std::string, const std::function<HashReturnType(const std:
class HashContent : public core::ProcessorImpl {
public:
- explicit HashContent(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<HashContent>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "HashContent calculates the checksum of the content of the flowfile and adds it as an attribute. "
"Configuration options exist to select hashing algorithm and set the name of the attribute.";
diff --git a/extensions/standard-processors/processors/InvokeHTTP.cpp b/extensions/standard-processors/processors/InvokeHTTP.cpp
index d7226ba64..084065cd4 100644
--- a/extensions/standard-processors/processors/InvokeHTTP.cpp
+++ b/extensions/standard-processors/processors/InvokeHTTP.cpp
@@ -204,7 +204,7 @@ void InvokeHTTP::onSchedule(core::ProcessContext& context, core::ProcessSessionF
return createHTTPClientFromMembers(url);
};
- client_queue_ = std::make_unique<invoke_http::HttpClientStore>(getMaxConcurrentTasks() * 2, create_client);
+ client_queue_ = std::make_unique<invoke_http::HttpClientStore>(context.getMaxConcurrentTasks() * 2, create_client);
}
bool InvokeHTTP::shouldEmitFlowFile() const {
@@ -255,7 +255,7 @@ void InvokeHTTP::onTrigger(core::ProcessContext& context, core::ProcessSession&
flow_file = session.create();
} else {
logger_->log_debug("Exiting because method is {} and there is no flowfile available to execute it, yielding", magic_enum::enum_name(method_));
- yield();
+ context.yield();
return;
}
} else {
diff --git a/extensions/standard-processors/processors/InvokeHTTP.h b/extensions/standard-processors/processors/InvokeHTTP.h
index 3d54491f4..da4652a89 100644
--- a/extensions/standard-processors/processors/InvokeHTTP.h
+++ b/extensions/standard-processors/processors/InvokeHTTP.h
@@ -28,7 +28,7 @@
#include "core/Core.h"
#include "core/OutputAttributeDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -127,10 +127,9 @@ class InvokeHTTP : public core::ProcessorImpl {
EXTENSIONAPI static constexpr std::string_view REQUEST_URL = "invokehttp.request.url";
EXTENSIONAPI static constexpr std::string_view TRANSACTION_ID = "invokehttp.tx.id";
- explicit InvokeHTTP(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<InvokeHTTP>::getLogger(uuid_);
- setTriggerWhenEmpty(true);
+ explicit InvokeHTTP(core::ProcessorMetadata info)
+ : ProcessorImpl(info) {
+ setTriggerWhenEmpty(true);
}
EXTENSIONAPI static constexpr const char* Description = "An HTTP client processor which can interact with a configurable HTTP Endpoint. "
diff --git a/extensions/standard-processors/processors/JoltTransformJSON.h b/extensions/standard-processors/processors/JoltTransformJSON.h
index 071f264ca..a21bc0df9 100644
--- a/extensions/standard-processors/processors/JoltTransformJSON.h
+++ b/extensions/standard-processors/processors/JoltTransformJSON.h
@@ -19,7 +19,7 @@
#include <string>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
@@ -38,11 +38,7 @@ namespace org::apache::nifi::minifi::processors {
class JoltTransformJSON : public core::ProcessorImpl {
public:
- explicit JoltTransformJSON(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<JoltTransformJSON>::getLogger(uuid_);
- }
-
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Applies a list of Jolt specifications to the flowfile JSON payload. A new FlowFile is created "
"with transformed content and is routed to the 'success' relationship. If the JSON transform "
diff --git a/extensions/standard-processors/processors/ListFile.h b/extensions/standard-processors/processors/ListFile.h
index e18d660ca..8a3037c4e 100644
--- a/extensions/standard-processors/processors/ListFile.h
+++ b/extensions/standard-processors/processors/ListFile.h
@@ -39,10 +39,7 @@ namespace org::apache::nifi::minifi::processors {
class ListFile : public core::ProcessorImpl {
public:
- explicit ListFile(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ListFile>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "Retrieves a listing of files from the local filesystem. For each file that is listed, "
"creates a FlowFile that represents the file so that it can be fetched in conjunction with FetchFile.";
diff --git a/extensions/standard-processors/processors/ListenSyslog.h b/extensions/standard-processors/processors/ListenSyslog.h
index 260f20f9d..5fb06dba2 100644
--- a/extensions/standard-processors/processors/ListenSyslog.h
+++ b/extensions/standard-processors/processors/ListenSyslog.h
@@ -37,9 +37,7 @@ namespace org::apache::nifi::minifi::processors {
class ListenSyslog : public NetworkListenerProcessor {
public:
- explicit ListenSyslog(std::string_view name, const utils::Identifier& uuid = {})
- : NetworkListenerProcessor(name, uuid, core::logging::LoggerFactory<ListenSyslog>::getLogger(uuid)) {
- }
+ using NetworkListenerProcessor::NetworkListenerProcessor;
EXTENSIONAPI static constexpr const char* Description = "Listens for Syslog messages being sent to a given port over TCP or UDP. "
"Incoming messages are optionally checked against regular expressions for RFC5424 and RFC3164 formatted messages. "
diff --git a/extensions/standard-processors/processors/ListenTCP.h b/extensions/standard-processors/processors/ListenTCP.h
index 4c6634996..d57058e1b 100644
--- a/extensions/standard-processors/processors/ListenTCP.h
+++ b/extensions/standard-processors/processors/ListenTCP.h
@@ -36,9 +36,7 @@ namespace org::apache::nifi::minifi::processors {
class ListenTCP : public NetworkListenerProcessor {
public:
- explicit ListenTCP(std::string_view name, const utils::Identifier& uuid = {})
- : NetworkListenerProcessor(name, uuid, core::logging::LoggerFactory<ListenTCP>::getLogger(uuid)) {
- }
+ using NetworkListenerProcessor::NetworkListenerProcessor;
EXTENSIONAPI static constexpr const char* Description = "Listens for incoming TCP connections and reads data from each connection using a configurable message delimiter. "
"For each message the processor produces a single FlowFile.";
diff --git a/extensions/standard-processors/processors/ListenUDP.h b/extensions/standard-processors/processors/ListenUDP.h
index 729f68a89..a5b075e31 100644
--- a/extensions/standard-processors/processors/ListenUDP.h
+++ b/extensions/standard-processors/processors/ListenUDP.h
@@ -31,9 +31,7 @@ namespace org::apache::nifi::minifi::processors {
class ListenUDP : public NetworkListenerProcessor {
public:
- explicit ListenUDP(const std::string& name, const utils::Identifier& uuid = {})
- : NetworkListenerProcessor(name, uuid, core::logging::LoggerFactory<ListenUDP>::getLogger(uuid)) {
- }
+ using NetworkListenerProcessor::NetworkListenerProcessor;
EXTENSIONAPI static constexpr const char* Description = "Listens for incoming UDP datagrams. For each datagram the processor produces a single FlowFile.";
diff --git a/extensions/standard-processors/processors/LogAttribute.h b/extensions/standard-processors/processors/LogAttribute.h
index 96b3a5fba..c47f414ff 100644
--- a/extensions/standard-processors/processors/LogAttribute.h
+++ b/extensions/standard-processors/processors/LogAttribute.h
@@ -25,7 +25,7 @@
#include <vector>
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
@@ -39,11 +39,11 @@ namespace org::apache::nifi::minifi::processors {
class LogAttribute : public core::ProcessorImpl {
public:
- explicit LogAttribute(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<LogAttribute>::getLogger(uuid_);
+ explicit LogAttribute(core::ProcessorMetadata info)
+ : ProcessorImpl(info) {
logger_->set_max_log_size(-1);
}
+
~LogAttribute() override = default;
EXTENSIONAPI static constexpr const char* Description = "Logs attributes of flow files in the MiNiFi application log.";
diff --git a/extensions/standard-processors/processors/NetworkListenerProcessor.h b/extensions/standard-processors/processors/NetworkListenerProcessor.h
index 2b4427cd0..624dfa591 100644
--- a/extensions/standard-processors/processors/NetworkListenerProcessor.h
+++ b/extensions/standard-processors/processors/NetworkListenerProcessor.h
@@ -20,7 +20,7 @@
#include <string>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/logging/Logger.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
@@ -31,10 +31,7 @@ namespace org::apache::nifi::minifi::processors {
class NetworkListenerProcessor : public core::ProcessorImpl {
public:
- NetworkListenerProcessor(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger)
- : core::ProcessorImpl(name, uuid),
- logger_(std::move(logger)) {
- }
+ using ProcessorImpl::ProcessorImpl;
~NetworkListenerProcessor() override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
@@ -64,9 +61,6 @@ class NetworkListenerProcessor : public core::ProcessorImpl {
std::string delimiter);
void startUdpServer(const core::ProcessContext& context);
- protected:
- std::shared_ptr<core::logging::Logger> logger_;
-
private:
struct ServerOptions {
std::optional<uint64_t> max_queue_size;
diff --git a/extensions/standard-processors/processors/PutFile.h b/extensions/standard-processors/processors/PutFile.h
index 8b5f35523..9b2de8570 100644
--- a/extensions/standard-processors/processors/PutFile.h
+++ b/extensions/standard-processors/processors/PutFile.h
@@ -20,7 +20,7 @@
#include <string>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
@@ -37,10 +37,7 @@ namespace org::apache::nifi::minifi::processors {
class PutFile : public core::ProcessorImpl {
public:
- explicit PutFile(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<PutFile>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~PutFile() override = default;
diff --git a/extensions/standard-processors/processors/PutTCP.cpp b/extensions/standard-processors/processors/PutTCP.cpp
index 7e0d1bcc1..f5a113238 100644
--- a/extensions/standard-processors/processors/PutTCP.cpp
+++ b/extensions/standard-processors/processors/PutTCP.cpp
@@ -38,11 +38,6 @@ namespace org::apache::nifi::minifi::processors {
constexpr size_t chunk_size = 1024;
-PutTCP::PutTCP(const std::string_view name, const utils::Identifier& uuid)
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<PutTCP>::getLogger(uuid_);
-}
-
PutTCP::~PutTCP() = default;
void PutTCP::initialize() {
@@ -54,10 +49,10 @@ void PutTCP::notifyStop() {}
void PutTCP::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) {
// if the required properties are missing or empty even before evaluating the EL expression, then we can throw in onSchedule, before we waste any flow files
- if (!getProperty(Hostname.name)) {
+ if (!context.hasNonEmptyProperty(Hostname.name)) {
throw Exception{ExceptionType::PROCESSOR_EXCEPTION, "missing hostname"};
}
- if (!getProperty(Port.name)) {
+ if (!context.hasNonEmptyProperty(Port.name)) {
throw Exception{ExceptionType::PROCESSOR_EXCEPTION, "missing port"};
}
if (const auto idle_connection_expiration = utils::parseOptionalDurationProperty(context, IdleConnectionExpiration); idle_connection_expiration && *idle_connection_expiration > 0ms)
@@ -97,7 +92,7 @@ void PutTCP::onSchedule(core::ProcessContext& context, core::ProcessSessionFacto
void PutTCP::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
const auto flow_file = session.get();
if (!flow_file) {
- yield();
+ context.yield();
return;
}
diff --git a/extensions/standard-processors/processors/PutTCP.h b/extensions/standard-processors/processors/PutTCP.h
index 1b574033f..71ea6858c 100644
--- a/extensions/standard-processors/processors/PutTCP.h
+++ b/extensions/standard-processors/processors/PutTCP.h
@@ -127,7 +127,7 @@ class PutTCP final : public core::ProcessorImpl {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PutTCP(const std::string_view name, const utils::Identifier& uuid = {});
+ using ProcessorImpl::ProcessorImpl;
PutTCP(const PutTCP&) = delete;
PutTCP(PutTCP&&) = delete;
PutTCP& operator=(const PutTCP&) = delete;
diff --git a/extensions/standard-processors/processors/PutUDP.cpp b/extensions/standard-processors/processors/PutUDP.cpp
index 03499fbef..f06c112c0 100644
--- a/extensions/standard-processors/processors/PutUDP.cpp
+++ b/extensions/standard-processors/processors/PutUDP.cpp
@@ -32,10 +32,6 @@ using asio::ip::udp;
namespace org::apache::nifi::minifi::processors {
-PutUDP::PutUDP(std::string_view name, const utils::Identifier& uuid)
- : ProcessorImpl(name, uuid), logger_{core::logging::LoggerFactory<PutUDP>::getLogger(uuid)}
-{ }
-
PutUDP::~PutUDP() = default;
void PutUDP::initialize() {
@@ -47,10 +43,10 @@ void PutUDP::notifyStop() {}
void PutUDP::onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) {
// if the required properties are missing or empty even before evaluating the EL expression, then we can throw in onSchedule, before we waste any flow files
- if (!context.getProperty(Hostname)) {
+ if (!context.hasNonEmptyProperty(Hostname.name)) {
throw Exception{ExceptionType::PROCESSOR_EXCEPTION, "missing hostname"};
}
- if (!context.getProperty(Port)) {
+ if (!context.hasNonEmptyProperty(Port.name)) {
throw Exception{ExceptionType::PROCESSOR_EXCEPTION, "missing port"};
}
}
@@ -58,7 +54,7 @@ void PutUDP::onSchedule(core::ProcessContext& context, core::ProcessSessionFacto
void PutUDP::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
const auto flow_file = session.get();
if (!flow_file) {
- yield();
+ context.yield();
return;
}
diff --git a/extensions/standard-processors/processors/PutUDP.h b/extensions/standard-processors/processors/PutUDP.h
index 10a5e592a..e4d7bf850 100644
--- a/extensions/standard-processors/processors/PutUDP.h
+++ b/extensions/standard-processors/processors/PutUDP.h
@@ -62,7 +62,7 @@ class PutUDP final : public core::ProcessorImpl {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit PutUDP(std::string_view name, const utils::Identifier& uuid = {});
+ using ProcessorImpl::ProcessorImpl;
PutUDP(const PutUDP&) = delete;
PutUDP& operator=(const PutUDP&) = delete;
~PutUDP() final;
@@ -71,8 +71,5 @@ class PutUDP final : public core::ProcessorImpl {
void notifyStop() final;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) final;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) final;
-
- private:
- std::shared_ptr<core::logging::Logger> logger_;
};
} // namespace org::apache::nifi::minifi::processors
diff --git a/extensions/standard-processors/processors/ReplaceText.cpp b/extensions/standard-processors/processors/ReplaceText.cpp
index dbb66a710..421811d22 100644
--- a/extensions/standard-processors/processors/ReplaceText.cpp
+++ b/extensions/standard-processors/processors/ReplaceText.cpp
@@ -28,11 +28,6 @@
namespace org::apache::nifi::minifi::processors {
-ReplaceText::ReplaceText(std::string_view name, const utils::Identifier& uuid)
- : core::ProcessorImpl(name, uuid),
- logger_(core::logging::LoggerFactory<ReplaceText>::getLogger(uuid)) {
-}
-
void ReplaceText::initialize() {
setSupportedProperties(Properties);
setSupportedRelationships(Relationships);
@@ -55,7 +50,7 @@ void ReplaceText::onTrigger(core::ProcessContext& context, core::ProcessSession&
std::shared_ptr<core::FlowFile> flow_file = session.get();
if (!flow_file) {
logger_->log_trace("No flow file");
- yield();
+ context.yield();
return;
}
@@ -77,7 +72,7 @@ ReplaceText::Parameters ReplaceText::readParameters(core::ProcessContext& contex
Parameters parameters;
const auto search_value = (replacement_strategy_ == ReplacementStrategyType::REGEX_REPLACE ?
- getProperty(SearchValue.name) : context.getProperty(SearchValue, flow_file.get()));
+ context.getRawProperty(SearchValue.name) : context.getProperty(SearchValue, flow_file.get()));
if (search_value) {
parameters.search_value_ = *search_value;
logger_->log_debug("the {} property is set to {}", SearchValue.name, parameters.search_value_);
@@ -90,7 +85,7 @@ ReplaceText::Parameters ReplaceText::readParameters(core::ProcessContext& contex
}
const auto replacement_value = (replacement_strategy_ == ReplacementStrategyType::REGEX_REPLACE ?
- getProperty(ReplacementValue.name) : context.getProperty(ReplacementValue, flow_file.get()));
+ context.getRawProperty(ReplacementValue.name) : context.getProperty(ReplacementValue, flow_file.get()));
if (replacement_value) {
parameters.replacement_value_ = *replacement_value;
logger_->log_debug("the {} property is set to {}", ReplacementValue.name, parameters.replacement_value_);
diff --git a/extensions/standard-processors/processors/ReplaceText.h b/extensions/standard-processors/processors/ReplaceText.h
index 2e2258c32..2d3dde34f 100644
--- a/extensions/standard-processors/processors/ReplaceText.h
+++ b/extensions/standard-processors/processors/ReplaceText.h
@@ -22,7 +22,7 @@
#include <utility>
#include "core/Annotation.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
@@ -115,6 +115,8 @@ namespace org::apache::nifi::minifi::processors {
class ReplaceText : public core::ProcessorImpl {
public:
+ using ProcessorImpl::ProcessorImpl;
+
EXTENSIONAPI static constexpr const char* Description = "Updates the content of a FlowFile by replacing parts of it using various replacement strategies.";
EXTENSIONAPI static constexpr auto EvaluationMode = core::PropertyDefinitionBuilder<magic_enum::enum_count<EvaluationModeType>()>::createProperty("Evaluation Mode")
@@ -177,7 +179,6 @@ class ReplaceText : public core::ProcessorImpl {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit ReplaceText(std::string_view name, const utils::Identifier& uuid = {});
void initialize() override;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory&) override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
@@ -204,7 +205,6 @@ class ReplaceText : public core::ProcessorImpl {
EvaluationModeType evaluation_mode_ = EvaluationModeType::LINE_BY_LINE;
LineByLineEvaluationModeType line_by_line_evaluation_mode_ = LineByLineEvaluationModeType::ALL;
ReplacementStrategyType replacement_strategy_ = ReplacementStrategyType::REGEX_REPLACE;
- std::shared_ptr<core::logging::Logger> logger_;
};
} // namespace org::apache::nifi::minifi::processors
diff --git a/extensions/standard-processors/processors/RetryFlowFile.h b/extensions/standard-processors/processors/RetryFlowFile.h
index abdb88d6b..b6db59d84 100644
--- a/extensions/standard-processors/processors/RetryFlowFile.h
+++ b/extensions/standard-processors/processors/RetryFlowFile.h
@@ -42,10 +42,7 @@ namespace org::apache::nifi::minifi::processors {
class RetryFlowFile : public core::ProcessorImpl {
public:
- explicit RetryFlowFile(const std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<RetryFlowFile>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~RetryFlowFile() override = default;
// ReuseMode allowed values
diff --git a/extensions/standard-processors/processors/RouteOnAttribute.cpp b/extensions/standard-processors/processors/RouteOnAttribute.cpp
index 573634be9..1e0fbbf31 100644
--- a/extensions/standard-processors/processors/RouteOnAttribute.cpp
+++ b/extensions/standard-processors/processors/RouteOnAttribute.cpp
@@ -45,7 +45,6 @@ void RouteOnAttribute::onSchedule(core::ProcessContext& context, core::ProcessSe
relationships.push_back(route_rel);
logger_->log_info("RouteOnAttribute registered route '{}' with expression '{}'", route.first, route.second);
}
- setSupportedRelationships(relationships);
}
void RouteOnAttribute::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
diff --git a/extensions/standard-processors/processors/RouteOnAttribute.h b/extensions/standard-processors/processors/RouteOnAttribute.h
index f1775bab7..92fd057f8 100644
--- a/extensions/standard-processors/processors/RouteOnAttribute.h
+++ b/extensions/standard-processors/processors/RouteOnAttribute.h
@@ -24,7 +24,7 @@
#include <string>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/RelationshipDefinition.h"
diff --git a/extensions/standard-processors/processors/RouteText.cpp b/extensions/standard-processors/processors/RouteText.cpp
index 205ec5566..a8b5c5aad 100644
--- a/extensions/standard-processors/processors/RouteText.cpp
+++ b/extensions/standard-processors/processors/RouteText.cpp
@@ -39,9 +39,6 @@
namespace org::apache::nifi::minifi::processors {
-RouteText::RouteText(std::string_view name, const utils::Identifier& uuid)
- : core::ProcessorImpl(name, uuid), logger_(core::logging::LoggerFactory<RouteText>::getLogger(uuid)) {}
-
void RouteText::initialize() {
setSupportedProperties(Properties);
setSupportedRelationships(Relationships);
@@ -61,14 +58,12 @@ void RouteText::onSchedule(core::ProcessContext& context, core::ProcessSessionFa
const auto static_relationships = RouteText::Relationships;
std::vector<core::RelationshipDefinition> relationships(static_relationships.begin(), static_relationships.end());
- for (const auto& property_name : getDynamicPropertyKeys()) {
+ for (const auto& property_name : context.getDynamicPropertyKeys()) {
core::RelationshipDefinition rel{property_name, "Dynamic Route"};
dynamic_relationships_[property_name] = rel;
relationships.push_back(rel);
logger_->log_info("RouteText registered dynamic route '{}'", property_name);
}
-
- setSupportedRelationships(relationships);
}
}
@@ -249,7 +244,7 @@ void RouteText::onTrigger(core::ProcessContext& context, core::ProcessSession& s
// group extraction always uses the preprocessed
auto group = getGroup(preprocessed_value);
- auto dynamic_property_keys = getDynamicPropertyKeys();
+ auto dynamic_property_keys = context.getDynamicPropertyKeys();
switch (routing_) {
case route_text::Routing::ALL: {
if (ranges::all_of(dynamic_property_keys, [&] (const auto& property_name) {
diff --git a/extensions/standard-processors/processors/RouteText.h b/extensions/standard-processors/processors/RouteText.h
index 811baa71e..d176f7a2c 100644
--- a/extensions/standard-processors/processors/RouteText.h
+++ b/extensions/standard-processors/processors/RouteText.h
@@ -24,7 +24,7 @@
#include <memory>
#include "core/OutputAttributeDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -205,7 +205,7 @@ class RouteText : public core::ProcessorImpl {
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit RouteText(std::string_view name, const utils::Identifier& uuid = {});
+ using ProcessorImpl::ProcessorImpl;
void initialize() override;
void onSchedule(core::ProcessContext& context, core::ProcessSessionFactory& session_factory) override;
@@ -236,8 +236,6 @@ class RouteText : public core::ProcessorImpl {
std::string group_fallback_;
std::map<std::string, core::Relationship> dynamic_relationships_;
-
- std::shared_ptr<core::logging::Logger> logger_;
};
} // namespace org::apache::nifi::minifi::processors
diff --git a/extensions/standard-processors/processors/SegmentContent.h b/extensions/standard-processors/processors/SegmentContent.h
index 364bfbbd8..203deb2b3 100644
--- a/extensions/standard-processors/processors/SegmentContent.h
+++ b/extensions/standard-processors/processors/SegmentContent.h
@@ -24,7 +24,7 @@
#include "FlowFileRecord.h"
#include "core/ProcessSession.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/RelationshipDefinition.h"
diff --git a/extensions/standard-processors/processors/SplitContent.h b/extensions/standard-processors/processors/SplitContent.h
index a07921220..8c26e7b7d 100644
--- a/extensions/standard-processors/processors/SplitContent.h
+++ b/extensions/standard-processors/processors/SplitContent.h
@@ -23,7 +23,7 @@
#include "FlowFileRecord.h"
#include "core/ProcessSession.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -34,9 +34,7 @@ namespace org::apache::nifi::minifi::processors {
class SplitContent final : public core::ProcessorImpl {
public:
- explicit SplitContent(const std::string_view name, const utils::Identifier& uuid = {}) : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<SplitContent>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
using size_type = std::vector<std::byte>::size_type;
enum class ByteSequenceFormat { Hexadecimal, Text };
diff --git a/extensions/standard-processors/processors/SplitRecord.cpp b/extensions/standard-processors/processors/SplitRecord.cpp
index 9d4d80e4b..56099608c 100644
--- a/extensions/standard-processors/processors/SplitRecord.cpp
+++ b/extensions/standard-processors/processors/SplitRecord.cpp
@@ -57,7 +57,7 @@ nonstd::expected<std::size_t, std::string> SplitRecord::readRecordsPerSplit(core
void SplitRecord::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
const auto original_flow_file = session.get();
if (!original_flow_file) {
- yield();
+ context.yield();
return;
}
diff --git a/extensions/standard-processors/processors/SplitText.h b/extensions/standard-processors/processors/SplitText.h
index 93f9f82a2..1be15c901 100644
--- a/extensions/standard-processors/processors/SplitText.h
+++ b/extensions/standard-processors/processors/SplitText.h
@@ -24,7 +24,7 @@
#include <utility>
#include <optional>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinitionBuilder.h"
#include "core/PropertyDefinition.h"
diff --git a/extensions/standard-processors/processors/TailFile.cpp b/extensions/standard-processors/processors/TailFile.cpp
index 708cf499e..2e61155c1 100644
--- a/extensions/standard-processors/processors/TailFile.cpp
+++ b/extensions/standard-processors/processors/TailFile.cpp
@@ -484,7 +484,7 @@ bool TailFile::getStateFromLegacyStateFile(const core::ProcessContext& context,
}
void TailFile::logState() {
- logger_->log_info("State of the TailFile processor {}:", name_);
+ logger_->log_info("State of the TailFile processor {}:", getName());
for (const auto& [key, value] : tail_states_) {
logger_->log_info("key => {{{}}}", value);
}
@@ -610,7 +610,7 @@ void TailFile::onTrigger(core::ProcessContext& context, core::ProcessSession& se
}
if (!session.existsFlowFileInRelationship(Success)) {
- yield();
+ context.yield();
}
first_trigger_ = false;
diff --git a/extensions/standard-processors/processors/TailFile.h b/extensions/standard-processors/processors/TailFile.h
index 3d8fa3a83..0095770da 100644
--- a/extensions/standard-processors/processors/TailFile.h
+++ b/extensions/standard-processors/processors/TailFile.h
@@ -29,7 +29,7 @@
#include "controllers/AttributeProviderService.h"
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
@@ -121,10 +121,7 @@ enum class Mode {
class TailFile : public core::ProcessorImpl {
public:
- explicit TailFile(const std::string_view name, const utils::Identifier& uuid = {})
- : core::ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<TailFile>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "\"Tails\" a file, or a list of files, ingesting data from the file as it is written to the file. The file is expected to be textual."
" Data is ingested only when a new line is encountered (carriage return or new-line character or combination). If the file to tail is periodically \"rolled over\","
diff --git a/extensions/standard-processors/processors/UpdateAttribute.cpp b/extensions/standard-processors/processors/UpdateAttribute.cpp
index 2c6f30df3..1076bad24 100644
--- a/extensions/standard-processors/processors/UpdateAttribute.cpp
+++ b/extensions/standard-processors/processors/UpdateAttribute.cpp
@@ -51,7 +51,7 @@ void UpdateAttribute::onTrigger(core::ProcessContext& context, core::ProcessSess
} catch (const std::exception &e) {
logger_->log_error("Caught exception while updating attributes: {}", e.what());
session.transfer(flow_file, Failure);
- yield();
+ context.yield();
}
}
diff --git a/extensions/standard-processors/processors/UpdateAttribute.h b/extensions/standard-processors/processors/UpdateAttribute.h
index e877b0669..df885ef35 100644
--- a/extensions/standard-processors/processors/UpdateAttribute.h
+++ b/extensions/standard-processors/processors/UpdateAttribute.h
@@ -24,7 +24,7 @@
#include <utility>
#include <vector>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/RelationshipDefinition.h"
diff --git a/extensions/standard-processors/tests/integration/InvokeHTTPTests.cpp b/extensions/standard-processors/tests/integration/InvokeHTTPTests.cpp
index 9e104f799..de591b2ae 100644
--- a/extensions/standard-processors/tests/integration/InvokeHTTPTests.cpp
+++ b/extensions/standard-processors/tests/integration/InvokeHTTPTests.cpp
@@ -140,7 +140,7 @@ TEST_CASE("HTTPTestsPenalizeNoRetry", "[httptest1]") {
TEST_CASE("InvokeHTTP fails with when flow contains invalid attribute names in HTTP headers", "[httptest1]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invokehttp = test_controller.getProcessor();
TestHTTPServer http_server;
@@ -162,7 +162,7 @@ TEST_CASE("InvokeHTTP succeeds when the flow file contains an attribute that wou
"[httptest1][invokehttp][httpheader][attribute]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invokehttp = test_controller.getProcessor();
TestHTTPServer http_server;
@@ -184,7 +184,7 @@ TEST_CASE("InvokeHTTP succeeds when the flow file contains an attribute that wou
TEST_CASE("InvokeHTTP replaces invalid characters of attributes", "[httptest1]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invokehttp = test_controller.getProcessor();
TestHTTPServer http_server;
@@ -205,7 +205,7 @@ TEST_CASE("InvokeHTTP replaces invalid characters of attributes", "[httptest1]")
TEST_CASE("InvokeHTTP drops invalid attributes from HTTP headers", "[httptest1]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invokehttp = test_controller.getProcessor();
TestHTTPServer http_server;
@@ -227,7 +227,7 @@ TEST_CASE("InvokeHTTP drops invalid attributes from HTTP headers", "[httptest1]"
TEST_CASE("InvokeHTTP empty Attributes to Send means no attributes are sent", "[httptest1]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invokehttp = test_controller.getProcessor();
TestHTTPServer http_server;
@@ -250,7 +250,7 @@ TEST_CASE("InvokeHTTP empty Attributes to Send means no attributes are sent", "[
TEST_CASE("InvokeHTTP DateHeader", "[InvokeHTTP]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invoke_http = test_controller.getProcessor();
TestHTTPServer http_server;
@@ -284,7 +284,7 @@ TEST_CASE("InvokeHTTP DateHeader", "[InvokeHTTP]") {
TEST_CASE("InvokeHTTP Attributes to Send uses full string matching, not substring", "[httptest1]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invokehttp = test_controller.getProcessor();
TestHTTPServer http_server;
@@ -307,7 +307,7 @@ TEST_CASE("InvokeHTTP Attributes to Send uses full string matching, not substrin
TEST_CASE("HTTPTestsResponseBodyinAttribute", "[InvokeHTTP]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invoke_http = test_controller.getProcessor();
minifi::test::ConnectionCountingServer connection_counting_server;
@@ -332,7 +332,7 @@ TEST_CASE("HTTPTestsResponseBodyinAttribute", "[InvokeHTTP]") {
TEST_CASE("HTTPTestsResponseBody", "[InvokeHTTP]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invoke_http = test_controller.getProcessor();
minifi::test::ConnectionCountingServer connection_counting_server;
@@ -355,7 +355,7 @@ TEST_CASE("HTTPTestsResponseBody", "[InvokeHTTP]") {
TEST_CASE("Test Keepalive", "[InvokeHTTP]") {
using minifi::processors::InvokeHTTP;
- test::SingleProcessorTestController test_controller{std::make_unique<InvokeHTTP>("InvokeHTTP")};
+ test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<InvokeHTTP>("InvokeHTTP")};
auto invoke_http = test_controller.getProcessor();
minifi::test::ConnectionCountingServer connection_counting_server;
diff --git a/extensions/standard-processors/tests/unit/AttributeRollingWindowTests.cpp b/extensions/standard-processors/tests/unit/AttributeRollingWindowTests.cpp
index f4a5ca6e0..ba3680c4b 100644
--- a/extensions/standard-processors/tests/unit/AttributeRollingWindowTests.cpp
+++ b/extensions/standard-processors/tests/unit/AttributeRollingWindowTests.cpp
@@ -21,6 +21,7 @@
#include "AttributeRollingWindow.h"
#include "unit/SingleProcessorTestController.h"
#include "core/FlowFile.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::test {
using AttributeRollingWindow = processors::AttributeRollingWindow;
@@ -36,7 +37,7 @@ bool checkAttributes(const std::map<std::string, std::string>& expected, const s
}
TEST_CASE("AttributeRollingWindow properly forwards properties to RollingWindow and sets attributes", "[attributerollingwindow]") {
- SingleProcessorTestController controller{std::make_unique<AttributeRollingWindow>("AttributeRollingWindow")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<AttributeRollingWindow>("AttributeRollingWindow")};
const auto proc = controller.getProcessor();
controller.plan->setProperty(proc, AttributeRollingWindow::ValueToTrack, "${value}");
controller.plan->setProperty(proc, AttributeRollingWindow::WindowLength, "3");
diff --git a/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp b/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
index 1eaa4ad30..49a22a8cc 100644
--- a/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
+++ b/extensions/standard-processors/tests/unit/DefragmentTextTests.cpp
@@ -37,10 +37,10 @@ namespace defragment_text = org::apache::nifi::minifi::processors::defragment_te
TEST_CASE("DefragmentText Single source tests", "[defragmenttextsinglesource]") {
TestController testController;
auto plan = testController.createPlan();
- auto write_to_flow_file = dynamic_cast<WriteToFlowFileTestProcessor*>(plan->addProcessor("WriteToFlowFileTestProcessor", "write_to_flow_file"));
- auto defrag_text_flow_files = dynamic_cast<DefragmentText*>(plan->addProcessor("DefragmentText", "defrag_text_flow_files"));
- auto read_from_success_relationship = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_success_relationship"));
- auto read_from_failure_relationship = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_failure_relationship"));
+ auto write_to_flow_file = plan->addProcessor<WriteToFlowFileTestProcessor>("write_to_flow_file");
+ auto defrag_text_flow_files = plan->addProcessor<DefragmentText>("defrag_text_flow_files");
+ auto read_from_success_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>( "read_from_success_relationship");
+ auto read_from_failure_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_failure_relationship");
plan->addConnection(write_to_flow_file, WriteToFlowFileTestProcessor::Success, defrag_text_flow_files);
@@ -65,77 +65,77 @@ TEST_CASE("DefragmentText Single source tests", "[defragmenttextsinglesource]")
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "<[0-9]+>");
plan->setProperty(defrag_text_flow_files, DefragmentText::PatternLoc, magic_enum::enum_name(defragment_text::PatternLocation::START_OF_MESSAGE));
- write_to_flow_file->setContent("<1> Foo");
+ write_to_flow_file.get().setContent("<1> Foo");
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
- write_to_flow_file->setContent("<2> Bar");
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
+ write_to_flow_file.get().setContent("<2> Bar");
plan->reset();
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("<1> Foo"));
- write_to_flow_file->setContent("<3> Baz");
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("<1> Foo"));
+ write_to_flow_file.get().setContent("<3> Baz");
plan->reset();
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("<2> Bar"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("<2> Bar"));
}
SECTION("Single line messages ending with pattern") {
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "<[0-9]+>");
plan->setProperty(defrag_text_flow_files, DefragmentText::PatternLoc, magic_enum::enum_name(defragment_text::PatternLocation::END_OF_MESSAGE));
- write_to_flow_file->setContent("Foo <1>");
+ write_to_flow_file.get().setContent("Foo <1>");
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("Foo <1>"));
- write_to_flow_file->setContent("Bar <2>");
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("Foo <1>"));
+ write_to_flow_file.get().setContent("Bar <2>");
plan->reset();
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("Bar <2>"));
- write_to_flow_file->setContent("Baz <3>");
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("Bar <2>"));
+ write_to_flow_file.get().setContent("Baz <3>");
plan->reset();
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("Baz <3>"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("Baz <3>"));
}
SECTION("Multiline matching start of messages") {
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "<[0-9]+>");
plan->setProperty(defrag_text_flow_files, DefragmentText::PatternLoc, magic_enum::enum_name(defragment_text::PatternLocation::START_OF_MESSAGE));
- write_to_flow_file->setContent("apple<1> banana<2> cherry<3> dragon ");
+ write_to_flow_file.get().setContent("apple<1> banana<2> cherry<3> dragon ");
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("apple<1> banana<2> cherry"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("apple<1> banana<2> cherry"));
- write_to_flow_file->setContent("fruit<4> elderberry<5> fig<6> grapefruit");
+ write_to_flow_file.get().setContent("fruit<4> elderberry<5> fig<6> grapefruit");
plan->reset();
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("<3> dragon fruit<4> elderberry<5> fig"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("<3> dragon fruit<4> elderberry<5> fig"));
}
SECTION("Multiline matching end of messages") {
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "<[0-9]+>");
plan->setProperty(defrag_text_flow_files, DefragmentText::PatternLoc, magic_enum::enum_name(defragment_text::PatternLocation::END_OF_MESSAGE));
- write_to_flow_file->setContent("apple<1> banana<2> cherry<3> dragon ");
+ write_to_flow_file.get().setContent("apple<1> banana<2> cherry<3> dragon ");
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("apple<1> banana<2> cherry<3>"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("apple<1> banana<2> cherry<3>"));
- write_to_flow_file->setContent("fruit<4> elderberry<5> fig<6> grapefruit");
+ write_to_flow_file.get().setContent("fruit<4> elderberry<5> fig<6> grapefruit");
plan->reset();
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent(" dragon fruit<4> elderberry<5> fig<6>"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent(" dragon fruit<4> elderberry<5> fig<6>"));
}
SECTION("Timeout test Start of Line") {
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "<[0-9]+>");
plan->setProperty(defrag_text_flow_files, DefragmentText::MaxBufferAge, "100 ms");
- write_to_flow_file->setContent("Message");
+ write_to_flow_file.get().setContent("Message");
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
plan->reset();
- write_to_flow_file->setContent("");
+ write_to_flow_file.get().setContent("");
std::this_thread::sleep_for(std::chrono::milliseconds(300));
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("Message"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("Message"));
}
SECTION("Timeout test Start of Line") {
@@ -143,15 +143,15 @@ TEST_CASE("DefragmentText Single source tests", "[defragmenttextsinglesource]")
plan->setProperty(defrag_text_flow_files, DefragmentText::PatternLoc, magic_enum::enum_name(defragment_text::PatternLocation::START_OF_MESSAGE));
plan->setProperty(defrag_text_flow_files, DefragmentText::MaxBufferAge, "100 ms");
- write_to_flow_file->setContent("Message");
+ write_to_flow_file.get().setContent("Message");
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
plan->reset();
- write_to_flow_file->setContent("");
+ write_to_flow_file.get().setContent("");
std::this_thread::sleep_for(std::chrono::milliseconds(300));
testController.runSession(plan);
- CHECK(read_from_success_relationship->readFlowFileWithContent("Message"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("Message"));
}
SECTION("Timeout test Start of Line") {
@@ -159,76 +159,76 @@ TEST_CASE("DefragmentText Single source tests", "[defragmenttextsinglesource]")
plan->setProperty(defrag_text_flow_files, DefragmentText::PatternLoc, magic_enum::enum_name(defragment_text::PatternLocation::END_OF_MESSAGE));
plan->setProperty(defrag_text_flow_files, DefragmentText::MaxBufferAge, "100 ms");
- write_to_flow_file->setContent("Message");
+ write_to_flow_file.get().setContent("Message");
testController.runSession(plan);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
plan->reset();
- write_to_flow_file->setContent("");
+ write_to_flow_file.get().setContent("");
std::this_thread::sleep_for(std::chrono::milliseconds(300));
testController.runSession(plan);
- CHECK(read_from_failure_relationship->readFlowFileWithContent("Message"));
+ CHECK(read_from_failure_relationship.get().readFlowFileWithContent("Message"));
}
SECTION("Timeout test without enough time") {
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "<[0-9]+>");
plan->setProperty(defrag_text_flow_files, DefragmentText::MaxBufferAge, "1 h");
- write_to_flow_file->setContent("Message");
+ write_to_flow_file.get().setContent("Message");
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
plan->reset();
- write_to_flow_file->setContent("");
+ write_to_flow_file.get().setContent("");
std::this_thread::sleep_for(std::chrono::milliseconds(300));
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
}
SECTION("Max Buffer test") {
plan->setProperty(defrag_text_flow_files, DefragmentText::MaxBufferSize, "100 B");
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "<[0-9]+>");
- write_to_flow_file->setContent("Message");
+ write_to_flow_file.get().setContent("Message");
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
plan->reset();
- write_to_flow_file->setContent(std::string(150, '*'));
+ write_to_flow_file.get().setContent(std::string(150, '*'));
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_failure_relationship->readFlowFileWithContent(std::string("Message").append(std::string(150, '*'))));
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure_relationship.get().readFlowFileWithContent(std::string("Message").append(std::string(150, '*'))));
}
SECTION("Max Buffer test without overflow") {
plan->setProperty(defrag_text_flow_files, DefragmentText::MaxBufferSize, "100 MB");
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "<[0-9]+>");
- write_to_flow_file->setContent("Message");
+ write_to_flow_file.get().setContent("Message");
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
plan->reset();
- write_to_flow_file->setContent(std::string(150, '*'));
+ write_to_flow_file.get().setContent(std::string(150, '*'));
testController.runSession(plan);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
}
}
TEST_CASE("DefragmentTextMultipleSources", "[defragmenttextinvalidsources]") {
TestController testController;
auto plan = testController.createPlan();
- auto input_1 = dynamic_cast<WriteToFlowFileTestProcessor*>(plan->addProcessor("WriteToFlowFileTestProcessor", "input_1"));
- auto input_2 = dynamic_cast<WriteToFlowFileTestProcessor*>(plan->addProcessor("WriteToFlowFileTestProcessor", "input_2"));
- auto update_ff_1 = dynamic_cast<UpdateAttribute*>(plan->addProcessor("UpdateAttribute", "update_attribute_1"));
- auto update_ff_2 = dynamic_cast<UpdateAttribute*>(plan->addProcessor("UpdateAttribute", "update_attribute_2"));
- auto defrag_text_flow_files = dynamic_cast<DefragmentText*>(plan->addProcessor("DefragmentText", "defrag_text_flow_files"));
- auto read_from_failure_relationship = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_failure_relationship"));
- auto read_from_success_relationship = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_success_relationship"));
+ auto input_1 = plan->addProcessor<WriteToFlowFileTestProcessor>("input_1");
+ auto input_2 = plan->addProcessor<WriteToFlowFileTestProcessor>("input_2");
+ auto update_ff_1 = plan->addProcessor<UpdateAttribute>("update_attribute_1");
+ auto update_ff_2 = plan->addProcessor<UpdateAttribute>("update_attribute_2");
+ auto defrag_text_flow_files = plan->addProcessor<DefragmentText>("defrag_text_flow_files");
+ auto read_from_failure_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_failure_relationship");
+ auto read_from_success_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_success_relationship");
plan->addConnection(input_1, WriteToFlowFileTestProcessor::Success, update_ff_1);
plan->addConnection(input_2, WriteToFlowFileTestProcessor::Success, update_ff_2);
@@ -238,8 +238,8 @@ TEST_CASE("DefragmentTextMultipleSources", "[defragmenttextinvalidsources]") {
plan->addConnection(defrag_text_flow_files, DefragmentText::Failure, read_from_failure_relationship);
plan->addConnection(defrag_text_flow_files, DefragmentText::Success, read_from_success_relationship);
- read_from_failure_relationship->disableClearOnTrigger();
- read_from_success_relationship->disableClearOnTrigger();
+ read_from_failure_relationship.get().disableClearOnTrigger();
+ read_from_success_relationship.get().disableClearOnTrigger();
read_from_failure_relationship->setAutoTerminatedRelationships(std::array<core::Relationship, 1>{ReadFromFlowFileTestProcessor::Success});
read_from_success_relationship->setAutoTerminatedRelationships(std::array<core::Relationship, 1>{ReadFromFlowFileTestProcessor::Success});
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "%");
@@ -248,66 +248,66 @@ TEST_CASE("DefragmentTextMultipleSources", "[defragmenttextinvalidsources]") {
plan->setDynamicProperty(update_ff_1, core::SpecialFlowAttribute::ABSOLUTE_PATH, "input_1");
plan->setDynamicProperty(update_ff_2, core::SpecialFlowAttribute::ABSOLUTE_PATH, "input_2");
- input_1->setContent("abc%def");
- input_2->setContent("ABC%DEF");
+ input_1.get().setContent("abc%def");
+ input_2.get().setContent("ABC%DEF");
testController.runSession(plan);
plan->reset();
- input_1->clearContent();
- input_2->clearContent();
+ input_1.get().clearContent();
+ input_2.get().clearContent();
testController.runSession(plan);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 2);
- CHECK(read_from_success_relationship->readFlowFileWithContent("abc"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("ABC"));
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 2);
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("abc"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("ABC"));
plan->reset();
- input_1->setContent("ghi%jkl");
- input_2->setContent("GHI%JKL");
+ input_1.get().setContent("ghi%jkl");
+ input_2.get().setContent("GHI%JKL");
testController.runSession(plan);
plan->reset();
- input_1->clearContent();
- input_2->clearContent();
+ input_1.get().clearContent();
+ input_2.get().clearContent();
testController.runSession(plan);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 4);
- CHECK(read_from_success_relationship->readFlowFileWithContent("%defghi"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("%DEFGHI"));
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 4);
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("%defghi"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("%DEFGHI"));
}
SECTION("Multiple Sources with same fragment attributes mix up") {
plan->setDynamicProperty(update_ff_1, core::SpecialFlowAttribute::ABSOLUTE_PATH, "input");
plan->setDynamicProperty(update_ff_2, core::SpecialFlowAttribute::ABSOLUTE_PATH, "input");
- input_1->setContent("abc%def");
- input_2->setContent("ABC%DEF");
+ input_1.get().setContent("abc%def");
+ input_2.get().setContent("ABC%DEF");
testController.runSession(plan);
plan->reset();
- input_1->clearContent();
- input_2->clearContent();
+ input_1.get().clearContent();
+ input_2.get().clearContent();
testController.runSession(plan);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 2);
- CHECK((read_from_success_relationship->readFlowFileWithContent("abc") || read_from_success_relationship->readFlowFileWithContent("ABC")));
- CHECK((read_from_success_relationship->readFlowFileWithContent("%DEFabc") || read_from_success_relationship->readFlowFileWithContent("%defABC")));
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 2);
+ CHECK((read_from_success_relationship.get().readFlowFileWithContent("abc") || read_from_success_relationship.get().readFlowFileWithContent("ABC")));
+ CHECK((read_from_success_relationship.get().readFlowFileWithContent("%DEFabc") || read_from_success_relationship.get().readFlowFileWithContent("%defABC")));
plan->reset();
- input_1->setContent("ghi%jkl");
- input_2->setContent("GHI%JKL");
+ input_1.get().setContent("ghi%jkl");
+ input_2.get().setContent("GHI%JKL");
testController.runSession(plan);
plan->reset();
- input_1->clearContent();
- input_2->clearContent();
+ input_1.get().clearContent();
+ input_2.get().clearContent();
testController.runSession(plan);
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 4);
- CHECK((read_from_success_relationship->readFlowFileWithContent("%defghi")
- || read_from_success_relationship->readFlowFileWithContent("%defGHI")
- || read_from_success_relationship->readFlowFileWithContent("%DEFGHI")
- || read_from_success_relationship->readFlowFileWithContent("%DEFghi")));
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 4);
+ CHECK((read_from_success_relationship.get().readFlowFileWithContent("%defghi")
+ || read_from_success_relationship.get().readFlowFileWithContent("%defGHI")
+ || read_from_success_relationship.get().readFlowFileWithContent("%DEFGHI")
+ || read_from_success_relationship.get().readFlowFileWithContent("%DEFghi")));
}
}
@@ -323,8 +323,8 @@ class FragmentGenerator : public core::ProcessorImpl {
static constexpr bool IsSingleThreaded = false;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
- explicit FragmentGenerator(std::string_view name, const utils::Identifier& uuid = utils::Identifier())
- : ProcessorImpl(name, uuid) {
+ explicit FragmentGenerator(minifi::core::ProcessorMetadata info)
+ : ProcessorImpl(info) {
}
void onTrigger(core::ProcessContext&, core::ProcessSession& session) override {
@@ -370,12 +370,12 @@ REGISTER_RESOURCE(FragmentGenerator, Processor);
TEST_CASE("DefragmentText with offset attributes", "[defragmenttextoffsetattributes]") {
TestController testController;
auto plan = testController.createPlan();
- auto input_1 = dynamic_cast<FragmentGenerator*>(plan->addProcessor("FragmentGenerator", "input_1"));
- auto input_2 = dynamic_cast<FragmentGenerator*>(plan->addProcessor("FragmentGenerator", "input_2"));
+ auto input_1 = plan->addProcessor<FragmentGenerator>("input_1");
+ auto input_2 = plan->addProcessor<FragmentGenerator>("input_2");
- auto defrag_text_flow_files = dynamic_cast<DefragmentText*>(plan->addProcessor("DefragmentText", "defrag_text_flow_files"));
- auto read_from_failure_relationship = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_failure_relationship"));
- auto read_from_success_relationship = dynamic_cast<ReadFromFlowFileTestProcessor*>(plan->addProcessor("ReadFromFlowFileTestProcessor", "read_from_success_relationship"));
+ auto defrag_text_flow_files = plan->addProcessor<DefragmentText>("defrag_text_flow_files");
+ auto read_from_failure_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_failure_relationship");
+ auto read_from_success_relationship = plan->addProcessor<ReadFromFlowFileTestProcessor>("read_from_success_relationship");
plan->addConnection(input_1, FragmentGenerator::Success, defrag_text_flow_files);
plan->addConnection(input_2, FragmentGenerator::Success, defrag_text_flow_files);
@@ -383,45 +383,45 @@ TEST_CASE("DefragmentText with offset attributes", "[defragmenttextoffsetattribu
plan->addConnection(defrag_text_flow_files, DefragmentText::Failure, read_from_failure_relationship);
plan->addConnection(defrag_text_flow_files, DefragmentText::Success, read_from_success_relationship);
- read_from_failure_relationship->disableClearOnTrigger();
- read_from_success_relationship->disableClearOnTrigger();
+ read_from_failure_relationship.get().disableClearOnTrigger();
+ read_from_success_relationship.get().disableClearOnTrigger();
read_from_failure_relationship->setAutoTerminatedRelationships(std::array<core::Relationship, 1>{ReadFromFlowFileTestProcessor::Success});
read_from_success_relationship->setAutoTerminatedRelationships(std::array<core::Relationship, 1>{ReadFromFlowFileTestProcessor::Success});
plan->setProperty(defrag_text_flow_files, DefragmentText::Pattern, "%");
- input_1->setBaseNameAttribute("input_1");
- input_2->setBaseNameAttribute("input_2");
- input_1->setPostNameAttribute("log");
- input_2->setPostNameAttribute("log");
- input_1->setAbsolutePathAttribute("/tmp/input/input_1.log");
- input_2->setAbsolutePathAttribute("/tmp/input/input_2.log");
+ input_1.get().setBaseNameAttribute("input_1");
+ input_2.get().setBaseNameAttribute("input_2");
+ input_1.get().setPostNameAttribute("log");
+ input_2.get().setPostNameAttribute("log");
+ input_1.get().setAbsolutePathAttribute("/tmp/input/input_1.log");
+ input_2.get().setAbsolutePathAttribute("/tmp/input/input_2.log");
SECTION("Single source input with offsets") {
- input_1->setFragments({"foo%bar", "%baz,app", "le%"});
+ input_1.get().setFragments({"foo%bar", "%baz,app", "le%"});
for (size_t i=0; i < 10; ++i) {
testController.runSession(plan);
plan->reset();
}
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 3);
- CHECK(read_from_success_relationship->readFlowFileWithContent("foo"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("%bar"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("%baz,apple"));
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 3);
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("foo"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("%bar"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("%baz,apple"));
}
SECTION("Two input sources with offsets") {
- input_1->setFragments({"foo%bar", "%baz,app", "le%"});
- input_2->setFragments({"monkey%dog", "%cat,octopu", "s%"});
+ input_1.get().setFragments({"foo%bar", "%baz,app", "le%"});
+ input_2.get().setFragments({"monkey%dog", "%cat,octopu", "s%"});
for (size_t i=0; i < 10; ++i) {
testController.runSession(plan);
plan->reset();
}
- CHECK(read_from_failure_relationship->numberOfFlowFilesRead() == 0);
- CHECK(read_from_success_relationship->numberOfFlowFilesRead() == 6);
- CHECK(read_from_success_relationship->readFlowFileWithContent("foo"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("%bar"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("%baz,apple"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("monkey"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("%dog"));
- CHECK(read_from_success_relationship->readFlowFileWithContent("%cat,octopus"));
+ CHECK(read_from_failure_relationship.get().numberOfFlowFilesRead() == 0);
+ CHECK(read_from_success_relationship.get().numberOfFlowFilesRead() == 6);
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("foo"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("%bar"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("%baz,apple"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("monkey"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("%dog"));
+ CHECK(read_from_success_relationship.get().readFlowFileWithContent("%cat,octopus"));
}
}
diff --git a/extensions/standard-processors/tests/unit/ExtractTextTests.cpp b/extensions/standard-processors/tests/unit/ExtractTextTests.cpp
index ea3969464..07c8eb099 100644
--- a/extensions/standard-processors/tests/unit/ExtractTextTests.cpp
+++ b/extensions/standard-processors/tests/unit/ExtractTextTests.cpp
@@ -46,7 +46,7 @@ const char* TEST_ATTR = "ExtractedText";
TEST_CASE("Test creation of ExtractText", "[extracttextCreate]") {
TestController testController;
- auto processor = std::make_unique<org::apache::nifi::minifi::processors::ExtractText>("processorname");
+ auto processor = minifi::test::utils::make_processor<org::apache::nifi::minifi::processors::ExtractText>("processorname");
REQUIRE(processor->getName() == "processorname");
utils::Identifier processoruuid = processor->getUUID();
REQUIRE(processoruuid);
diff --git a/extensions/standard-processors/tests/unit/FetchFileTests.cpp b/extensions/standard-processors/tests/unit/FetchFileTests.cpp
index 864857d76..a8f1939a6 100644
--- a/extensions/standard-processors/tests/unit/FetchFileTests.cpp
+++ b/extensions/standard-processors/tests/unit/FetchFileTests.cpp
@@ -46,7 +46,7 @@ class FetchFileTestFixture {
[[nodiscard]] std::unordered_multiset<std::string> getDirContents(const std::filesystem::path& dir_path) const;
std::shared_ptr<minifi::test::SingleProcessorTestController> test_controller_;
- minifi::processors::FetchFile* fetch_file_processor_ = nullptr;
+ minifi::core::Processor* fetch_file_processor_ = nullptr;
const std::filesystem::path input_dir_;
const std::filesystem::path permission_denied_file_name_;
const std::filesystem::path input_file_name_;
@@ -55,8 +55,8 @@ class FetchFileTestFixture {
};
FetchFileTestFixture::FetchFileTestFixture()
- : test_controller_(std::make_shared<minifi::test::SingleProcessorTestController>(std::make_unique<minifi::processors::FetchFile>("FetchFile"))),
- fetch_file_processor_(test_controller_->getProcessor<minifi::processors::FetchFile>()),
+ : test_controller_(std::make_shared<minifi::test::SingleProcessorTestController>(minifi::test::utils::make_processor<minifi::processors::FetchFile>("FetchFile"))),
+ fetch_file_processor_(test_controller_->getProcessor()),
input_dir_(test_controller_->createTempDirectory()),
permission_denied_file_name_("permission_denied.txt"),
input_file_name_("test.txt"),
diff --git a/extensions/standard-processors/tests/unit/FlowJsonTests.cpp b/extensions/standard-processors/tests/unit/FlowJsonTests.cpp
index 88a3fd1f8..c36dc6664 100644
--- a/extensions/standard-processors/tests/unit/FlowJsonTests.cpp
+++ b/extensions/standard-processors/tests/unit/FlowJsonTests.cpp
@@ -157,19 +157,19 @@ TEST_CASE("NiFi flow json format is correctly parsed") {
CHECK(proc->getProperty("Batch Size") == "12");
// verify funnel
- auto* funnel = dynamic_cast<minifi::Funnel*>(flow->findProcessorByName("CoolFunnel"));
+ TypedProcessorWrapper<minifi::Funnel> funnel = flow->findProcessorByName("CoolFunnel");
REQUIRE(funnel);
CHECK(funnel->getUUIDStr() == "00000000-0000-0000-0000-000000000010");
// verify RPG input port
- auto* port = dynamic_cast<minifi::RemoteProcessorGroupPort*>(flow->findProcessorByName("AmazingInputPort"));
+ TypedProcessorWrapper<minifi::RemoteProcessorGroupPort> port = flow->findProcessorByName("AmazingInputPort");
REQUIRE(port);
CHECK(port->getUUIDStr() == "00000000-0000-0000-0000-000000000003");
CHECK(port->getMaxConcurrentTasks() == 7);
- CHECK(port->getInstances().size() == 1);
- CHECK(port->getInstances().front().host_ == "localhost");
- CHECK(port->getInstances().front().port_ == 8090);
- CHECK(port->getInstances().front().protocol_ == "https://");
+ CHECK(port.get().getInstances().size() == 1);
+ CHECK(port.get().getInstances().front().host_ == "localhost");
+ CHECK(port.get().getInstances().front().port_ == 8090);
+ CHECK(port.get().getInstances().front().protocol_ == "https://");
CHECK(port->getProperty("Port UUID") == "00000000-0000-0000-0000-000000000005");
// verify connection
@@ -753,7 +753,7 @@ TEST_CASE("Property value sequences can use parameters") {
std::unique_ptr<core::ProcessGroup> flow = config.getRootFromPayload(CONFIG_JSON);
REQUIRE(flow);
- auto* proc = dynamic_cast<core::ProcessorImpl*>(flow->findProcessorByName("MyProcessor"));
+ auto* proc = flow->findProcessorByName("MyProcessor");
REQUIRE(proc);
auto values = proc->getAllPropertyValues("Simple Property");
REQUIRE(values);
@@ -814,7 +814,7 @@ TEST_CASE("Dynamic properties can use parameters") {
std::unique_ptr<core::ProcessGroup> flow = config.getRootFromPayload(CONFIG_JSON);
REQUIRE(flow);
- auto* proc = dynamic_cast<core::ProcessorImpl*>(flow->findProcessorByName("MyProcessor"));
+ auto* proc = flow->findProcessorByName("MyProcessor");
REQUIRE(proc);
auto values = proc->getAllDynamicPropertyValues("My Dynamic Property Sequence");
REQUIRE(values);
@@ -930,7 +930,7 @@ TEST_CASE("Test sensitive parameters in sensitive property value sequence") {
std::unique_ptr<core::ProcessGroup> flow = config.getRootFromPayload(CONFIG_JSON);
REQUIRE(flow);
- auto* proc = dynamic_cast<core::ProcessorImpl*>(flow->findProcessorByName("MyProcessor"));
+ auto* proc = flow->findProcessorByName("MyProcessor");
REQUIRE(proc);
core::Property property("Sensitive Property", "");
auto values = proc->getAllPropertyValues("Sensitive Property");
@@ -1031,17 +1031,17 @@ TEST_CASE("NiFi flow json can use alternative targetUris field") {
REQUIRE(flow);
// verify RPG input port
- auto* port = dynamic_cast<minifi::RemoteProcessorGroupPort*>(flow->findProcessorByName("AmazingInputPort"));
+ TypedProcessorWrapper<minifi::RemoteProcessorGroupPort> port = flow->findProcessorByName("AmazingInputPort");
REQUIRE(port);
- CHECK(port->getUUIDStr() == "00000000-0000-0000-0000-000000000003");
+ CHECK(port.get().getUUIDStr() == "00000000-0000-0000-0000-000000000003");
CHECK(port->getMaxConcurrentTasks() == 7);
if (target_uri_valid) {
- CHECK(port->getInstances().size() == 1);
- CHECK(port->getInstances().front().host_ == "localhost");
- CHECK(port->getInstances().front().port_ == 8090);
- CHECK(port->getInstances().front().protocol_ == "https://");
+ CHECK(port.get().getInstances().size() == 1);
+ CHECK(port.get().getInstances().front().host_ == "localhost");
+ CHECK(port.get().getInstances().front().port_ == 8090);
+ CHECK(port.get().getInstances().front().protocol_ == "https://");
} else {
- CHECK(port->getInstances().empty());
+ CHECK(port.get().getInstances().empty());
}
CHECK(port->getProperty("Port UUID") == "00000000-0000-0000-0000-000000000005");
}
diff --git a/extensions/standard-processors/tests/unit/GenerateFlowFileTests.cpp b/extensions/standard-processors/tests/unit/GenerateFlowFileTests.cpp
index 7e9d428f6..0c8cd1ccb 100644
--- a/extensions/standard-processors/tests/unit/GenerateFlowFileTests.cpp
+++ b/extensions/standard-processors/tests/unit/GenerateFlowFileTests.cpp
@@ -23,6 +23,7 @@
#include "unit/SingleProcessorTestController.h"
#include "unit/Catch.h"
#include "GenerateFlowFile.h"
+#include "unit/TestUtils.h"
using minifi::processors::GenerateFlowFile;
@@ -38,7 +39,7 @@ TEST_CASE("GenerateFlowFileWithBinaryData") {
}
- minifi::test::SingleProcessorTestController test_controller{std::make_unique<GenerateFlowFile>("GenerateFlowFile")};
+ minifi::test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<GenerateFlowFile>("GenerateFlowFile")};
auto generate_flow_file = test_controller.getProcessor();
LogTestController::getInstance().setWarn<GenerateFlowFile>();
@@ -67,7 +68,7 @@ TEST_CASE("GenerateFlowFileWithBinaryData") {
}
TEST_CASE("GenerateFlowFileTestEmpty") {
- minifi::test::SingleProcessorTestController test_controller{std::make_unique<GenerateFlowFile>("GenerateFlowFile")};
+ minifi::test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<GenerateFlowFile>("GenerateFlowFile")};
auto generate_flow_file = test_controller.getProcessor();
CHECK(test_controller.plan->setProperty(generate_flow_file, GenerateFlowFile::FileSize, "0"));
@@ -81,7 +82,7 @@ TEST_CASE("GenerateFlowFileTestEmpty") {
}
TEST_CASE("GenerateFlowFileCustomTextTest") {
- minifi::test::SingleProcessorTestController test_controller{std::make_unique<GenerateFlowFile>("GenerateFlowFile")};
+ minifi::test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<GenerateFlowFile>("GenerateFlowFile")};
auto generate_flow_file = test_controller.getProcessor();
constexpr auto uuid_string_length = 36;
@@ -97,7 +98,7 @@ TEST_CASE("GenerateFlowFileCustomTextTest") {
}
TEST_CASE("GenerateFlowFileCustomTextEmptyTest") {
- minifi::test::SingleProcessorTestController test_controller{std::make_unique<GenerateFlowFile>("GenerateFlowFile")};
+ minifi::test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<GenerateFlowFile>("GenerateFlowFile")};
auto generate_flow_file = test_controller.getProcessor();
constexpr int32_t file_size = 10;
@@ -119,7 +120,7 @@ TEST_CASE("GenerateFlowFileCustomTextEmptyTest") {
}
TEST_CASE("GenerateFlowFile reevaluating CustomText") {
- minifi::test::SingleProcessorTestController test_controller{std::make_unique<GenerateFlowFile>("GenerateFlowFile")};
+ minifi::test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<GenerateFlowFile>("GenerateFlowFile")};
auto generate_flow_file = test_controller.getProcessor();
CHECK(test_controller.plan->setProperty(generate_flow_file, GenerateFlowFile::DataFormat, "Text"));
CHECK(test_controller.plan->setProperty(generate_flow_file, GenerateFlowFile::UniqueFlowFiles, "false"));
@@ -136,7 +137,7 @@ TEST_CASE("GenerateFlowFile reevaluating CustomText") {
}
TEST_CASE("GenerateFlowFile CustomText evaluates to empty string") {
- minifi::test::SingleProcessorTestController test_controller{std::make_unique<GenerateFlowFile>("GenerateFlowFile")};
+ minifi::test::SingleProcessorTestController test_controller{minifi::test::utils::make_processor<GenerateFlowFile>("GenerateFlowFile")};
auto generate_flow_file = test_controller.getProcessor();
CHECK(test_controller.plan->setProperty(generate_flow_file, GenerateFlowFile::DataFormat, "Text"));
CHECK(test_controller.plan->setProperty(generate_flow_file, GenerateFlowFile::UniqueFlowFiles, "false"));
diff --git a/extensions/standard-processors/tests/unit/GetFileTests.cpp b/extensions/standard-processors/tests/unit/GetFileTests.cpp
index 84faf010b..20e368530 100644
--- a/extensions/standard-processors/tests/unit/GetFileTests.cpp
+++ b/extensions/standard-processors/tests/unit/GetFileTests.cpp
@@ -265,7 +265,7 @@ TEST_CASE("GetFile sets attributes correctly") {
using minifi::processors::GetFile;
LogTestController::getInstance().setTrace<GetFile>();
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<GetFile>("GetFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<GetFile>("GetFile"));
const auto get_file = test_controller.getProcessor();
std::filesystem::path dir = test_controller.createTempDirectory();
REQUIRE(get_file->setProperty(GetFile::Directory.name, dir.string()));
@@ -297,7 +297,7 @@ TEST_CASE("GetFile can use expression language in Directory property") {
using minifi::processors::GetFile;
LogTestController::getInstance().setTrace<GetFile>();
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<GetFile>("GetFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<GetFile>("GetFile"));
const auto get_file = test_controller.getProcessor();
std::filesystem::path base_dir = test_controller.createTempDirectory();
diff --git a/extensions/standard-processors/tests/unit/GetTCPTests.cpp b/extensions/standard-processors/tests/unit/GetTCPTests.cpp
index d7fd4aaff..6dfc56e90 100644
--- a/extensions/standard-processors/tests/unit/GetTCPTests.cpp
+++ b/extensions/standard-processors/tests/unit/GetTCPTests.cpp
@@ -152,7 +152,7 @@ class TcpTestServer {
};
TEST_CASE("GetTCP test with delimiter", "[GetTCP]") {
- SingleProcessorTestController controller{std::make_unique<GetTCP>("GetTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<GetTCP>("GetTCP")};
const auto get_tcp = controller.getProcessor();
LogTestController::getInstance().setTrace<GetTCP>();
REQUIRE(get_tcp->setProperty(GetTCP::MaxBatchSize.name, "2"));
@@ -183,7 +183,7 @@ TEST_CASE("GetTCP test with delimiter", "[GetTCP]") {
}
TEST_CASE("GetTCP test with too large message", "[GetTCP]") {
- SingleProcessorTestController controller{std::make_unique<GetTCP>("GetTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<GetTCP>("GetTCP")};
const auto get_tcp = controller.getProcessor();
LogTestController::getInstance().setTrace<GetTCP>();
REQUIRE(get_tcp->setProperty(GetTCP::MaxBatchSize.name, "2"));
@@ -224,7 +224,7 @@ TEST_CASE("GetTCP test with too large message", "[GetTCP]") {
}
TEST_CASE("GetTCP test multiple endpoints", "[GetTCP]") {
- SingleProcessorTestController controller{std::make_unique<GetTCP>("GetTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<GetTCP>("GetTCP")};
const auto get_tcp = controller.getProcessor();
LogTestController::getInstance().setTrace<GetTCP>();
REQUIRE(get_tcp->setProperty(GetTCP::MaxBatchSize.name, "2"));
@@ -268,7 +268,7 @@ TEST_CASE("GetTCP test multiple endpoints", "[GetTCP]") {
}
TEST_CASE("GetTCP max queue and max batch size test", "[GetTCP]") {
- SingleProcessorTestController controller{std::make_unique<GetTCP>("GetTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<GetTCP>("GetTCP")};
const auto get_tcp = controller.getProcessor();
LogTestController::getInstance().setTrace<GetTCP>();
REQUIRE(get_tcp->setProperty(GetTCP::MaxBatchSize.name, "10"));
diff --git a/extensions/standard-processors/tests/unit/HashContentTest.cpp b/extensions/standard-processors/tests/unit/HashContentTest.cpp
index 1654faeef..a7f851047 100644
--- a/extensions/standard-processors/tests/unit/HashContentTest.cpp
+++ b/extensions/standard-processors/tests/unit/HashContentTest.cpp
@@ -49,7 +49,7 @@ namespace org::apache::nifi::minifi::processors::test {
TEST_CASE("Test Creation of HashContent", "[HashContentCreate]") {
TestController testController;
- auto processor = std::make_shared<org::apache::nifi::minifi::processors::HashContent>("processorname");
+ auto processor = minifi::test::utils::make_processor<org::apache::nifi::minifi::processors::HashContent>("processorname");
REQUIRE(processor->getName() == "processorname");
REQUIRE(processor->getUUID());
}
@@ -166,7 +166,7 @@ TEST_CASE("TestingFailOnEmptyProperty", "[HashContentPropertiesCheck]") {
}
TEST_CASE("Invalid hash algorithm throws in onSchedule", "[HashContent]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<HashContent>("HashContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<HashContent>("HashContent")};
auto hash_content = controller.getProcessor();
REQUIRE(hash_content->setProperty(HashContent::HashAlgorithm.name, "My-Algo"));
REQUIRE_THROWS_WITH(controller.plan->scheduleProcessor(hash_content), "Process Schedule Operation: MYALGO is not supported, supported algorithms are: MD5, SHA1, SHA256");
diff --git a/extensions/standard-processors/tests/unit/JoltTransformJSONTests.cpp b/extensions/standard-processors/tests/unit/JoltTransformJSONTests.cpp
index 02a4ec49e..3a96e5748 100644
--- a/extensions/standard-processors/tests/unit/JoltTransformJSONTests.cpp
+++ b/extensions/standard-processors/tests/unit/JoltTransformJSONTests.cpp
@@ -28,7 +28,7 @@ namespace org::apache::nifi::minifi::test {
// NOLINTBEGIN(readability-container-size-empty)
TEST_CASE("Shiftr successful case") {
- SingleProcessorTestController controller{std::make_unique<minifi::processors::JoltTransformJSON>("JoltProc")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::JoltTransformJSON>("JoltProc")};
auto proc = controller.getProcessor();
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltTransform, magic_enum::enum_name(processors::jolt_transform_json::JoltTransform::Shift));
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltSpecification, R"json(
@@ -76,7 +76,7 @@ TEST_CASE("Shiftr successful case") {
}
TEST_CASE("Shiftr multiple destination") {
- SingleProcessorTestController controller{std::make_unique<minifi::processors::JoltTransformJSON>("JoltProc")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::JoltTransformJSON>("JoltProc")};
auto proc = controller.getProcessor();
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltTransform, magic_enum::enum_name(processors::jolt_transform_json::JoltTransform::Shift));
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltSpecification, R"json(
@@ -128,7 +128,7 @@ TEST_CASE("Shiftr invalid reference") {
}
TEST_CASE("Shiftr matches are correctly ordered") {
- SingleProcessorTestController controller{std::make_unique<minifi::processors::JoltTransformJSON>("JoltProc")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::JoltTransformJSON>("JoltProc")};
auto proc = controller.getProcessor();
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltTransform, magic_enum::enum_name(processors::jolt_transform_json::JoltTransform::Shift));
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltSpecification, R"json(
@@ -172,7 +172,7 @@ TEST_CASE("Shiftr matches are correctly ordered") {
}
TEST_CASE("Shiftr arrays are maps with numeric keys") {
- SingleProcessorTestController controller{std::make_unique<minifi::processors::JoltTransformJSON>("JoltProc")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::JoltTransformJSON>("JoltProc")};
auto proc = controller.getProcessor();
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltTransform, magic_enum::enum_name(processors::jolt_transform_json::JoltTransform::Shift));
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltSpecification, R"json(
@@ -204,7 +204,7 @@ TEST_CASE("Shiftr arrays are maps with numeric keys") {
}
TEST_CASE("Shiftr put into array at index") {
- SingleProcessorTestController controller{std::make_unique<minifi::processors::JoltTransformJSON>("JoltProc")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::JoltTransformJSON>("JoltProc")};
auto proc = controller.getProcessor();
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltTransform, magic_enum::enum_name(processors::jolt_transform_json::JoltTransform::Shift));
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltSpecification, R"json(
@@ -238,7 +238,7 @@ TEST_CASE("Shiftr put into array at index") {
TEST_CASE("Shiftr multiple patterns") {
// this is an extension, so we can escape and match on a '|' character
- SingleProcessorTestController controller{std::make_unique<minifi::processors::JoltTransformJSON>("JoltProc")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::JoltTransformJSON>("JoltProc")};
auto proc = controller.getProcessor();
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltTransform, magic_enum::enum_name(processors::jolt_transform_json::JoltTransform::Shift));
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltSpecification, R"json(
@@ -305,7 +305,7 @@ TEST_CASE("Run tests from https://github.com/bazaarvoice/jolt") {
throw std::logic_error(fmt::format("Error in test json '{}' at {}:{} : {}", entry.string(), cursor.first, cursor.second, rapidjson::GetParseError_En(parse_res.Code())));
}
- SingleProcessorTestController controller{std::make_unique<minifi::processors::JoltTransformJSON>("JoltProc")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<minifi::processors::JoltTransformJSON>("JoltProc")};
auto proc = controller.getProcessor();
LogTestController::getInstance().setTrace<minifi::processors::JoltTransformJSON>();
controller.plan->setProperty(proc, processors::JoltTransformJSON::JoltTransform, magic_enum::enum_name(processors::jolt_transform_json::JoltTransform::Shift));
diff --git a/extensions/standard-processors/tests/unit/ListFileTests.cpp b/extensions/standard-processors/tests/unit/ListFileTests.cpp
index 077cb45a0..c27b13f1f 100644
--- a/extensions/standard-processors/tests/unit/ListFileTests.cpp
+++ b/extensions/standard-processors/tests/unit/ListFileTests.cpp
@@ -228,7 +228,7 @@ TEST_CASE("ListFile sets attributes correctly") {
using minifi::processors::ListFile;
LogTestController::getInstance().setTrace<ListFile>();
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<ListFile>("ListFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<ListFile>("ListFile"));
const auto list_file = test_controller.getProcessor();
std::filesystem::path dir = test_controller.createTempDirectory();
REQUIRE(list_file->setProperty(ListFile::InputDirectory.name, dir.string()));
@@ -256,7 +256,7 @@ TEST_CASE("ListFile sets attributes correctly") {
TEST_CASE("If a second file with the same modification time shows up later, then it will get listed") {
using minifi::processors::ListFile;
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<ListFile>("ListFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<ListFile>("ListFile"));
const auto list_file = test_controller.getProcessor();
const auto input_dir = test_controller.createTempDirectory();
diff --git a/extensions/standard-processors/tests/unit/ListenSyslogTests.cpp b/extensions/standard-processors/tests/unit/ListenSyslogTests.cpp
index b206fbd16..a8dfc1465 100644
--- a/extensions/standard-processors/tests/unit/ListenSyslogTests.cpp
+++ b/extensions/standard-processors/tests/unit/ListenSyslogTests.cpp
@@ -249,7 +249,7 @@ void check_parsed_attributes(const core::FlowFile& flow_file, const ValidRFC3164
}
TEST_CASE("ListenSyslog without parsing test", "[ListenSyslog]") {
- SingleProcessorTestController controller{std::make_unique<ListenSyslog>("ListenSyslog")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenSyslog>("ListenSyslog")};
const auto listen_syslog = controller.getProcessor<ListenSyslog>();
LogTestController::getInstance().setTrace<ListenSyslog>();
REQUIRE(listen_syslog->setProperty(ListenSyslog::MaxBatchSize.name, "2"));
@@ -261,7 +261,7 @@ TEST_CASE("ListenSyslog without parsing test", "[ListenSyslog]") {
REQUIRE(listen_syslog->setProperty(ListenSyslog::ProtocolProperty.name, "UDP"));
protocol = "UDP";
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_syslog);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_syslog);
asio::ip::udp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -281,7 +281,7 @@ TEST_CASE("ListenSyslog without parsing test", "[ListenSyslog]") {
REQUIRE(listen_syslog->setProperty(ListenSyslog::ProtocolProperty.name, "TCP"));
protocol = "TCP";
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_syslog);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_syslog);
asio::ip::tcp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -305,7 +305,7 @@ TEST_CASE("ListenSyslog without parsing test", "[ListenSyslog]") {
}
TEST_CASE("ListenSyslog with parsing test", "[ListenSyslog][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenSyslog>("ListenSyslog")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenSyslog>("ListenSyslog")};
const auto listen_syslog = controller.getProcessor<ListenSyslog>();
LogTestController::getInstance().setTrace<ListenSyslog>();
REQUIRE(listen_syslog->setProperty(ListenSyslog::MaxBatchSize.name, "100"));
@@ -317,7 +317,7 @@ TEST_CASE("ListenSyslog with parsing test", "[ListenSyslog][NetworkListenerProce
REQUIRE(listen_syslog->setProperty(ListenSyslog::ProtocolProperty.name, "UDP"));
protocol = "UDP";
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_syslog);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_syslog);
asio::ip::udp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -347,7 +347,7 @@ TEST_CASE("ListenSyslog with parsing test", "[ListenSyslog][NetworkListenerProce
REQUIRE(listen_syslog->setProperty(ListenSyslog::ProtocolProperty.name, "TCP"));
protocol = "TCP";
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_syslog);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_syslog);
asio::ip::tcp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -410,7 +410,7 @@ TEST_CASE("ListenSyslog with parsing test", "[ListenSyslog][NetworkListenerProce
}
TEST_CASE("ListenSyslog can be rescheduled", "[ListenSyslog][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenSyslog>("ListenSyslog")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenSyslog>("ListenSyslog")};
const auto listen_syslog = controller.getProcessor<ListenSyslog>();
LogTestController::getInstance().setTrace<ListenSyslog>();
REQUIRE(listen_syslog->setProperty(ListenSyslog::Port.name, "0"));
@@ -432,7 +432,7 @@ TEST_CASE("ListenSyslog can be rescheduled", "[ListenSyslog][NetworkListenerProc
}
TEST_CASE("ListenSyslog max queue and max batch size test", "[ListenSyslog][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenSyslog>("ListenSyslog")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenSyslog>("ListenSyslog")};
const auto listen_syslog = controller.getProcessor<ListenSyslog>();
REQUIRE(listen_syslog->setProperty(ListenSyslog::MaxBatchSize.name, "10"));
REQUIRE(listen_syslog->setProperty(ListenSyslog::ParseMessages.name, "false"));
@@ -444,7 +444,7 @@ TEST_CASE("ListenSyslog max queue and max batch size test", "[ListenSyslog][Netw
SECTION("UDP") {
REQUIRE(listen_syslog->setProperty(ListenSyslog::ProtocolProperty.name, "UDP"));
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_syslog);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_syslog);
asio::ip::udp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -463,7 +463,7 @@ TEST_CASE("ListenSyslog max queue and max batch size test", "[ListenSyslog][Netw
SECTION("TCP") {
REQUIRE(listen_syslog->setProperty(ListenSyslog::ProtocolProperty.name, "TCP"));
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_syslog);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_syslog);
asio::ip::tcp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -490,7 +490,7 @@ TEST_CASE("ListenSyslog max queue and max batch size test", "[ListenSyslog][Netw
}
TEST_CASE("Test ListenSyslog via TCP with SSL connection", "[ListenSyslog][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenSyslog>("ListenSyslog")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenSyslog>("ListenSyslog")};
const auto listen_syslog = controller.getProcessor<ListenSyslog>();
auto ssl_context_service = controller.plan->addController("SSLContextService", "SSLContextService");
@@ -506,7 +506,7 @@ TEST_CASE("Test ListenSyslog via TCP with SSL connection", "[ListenSyslog][Netwo
REQUIRE(listen_syslog->setProperty(ListenSyslog::ProtocolProperty.name, "TCP"));
REQUIRE(listen_syslog->setProperty(ListenSyslog::SSLContextService.name, "SSLContextService"));
- auto port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_syslog);
+ auto port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_syslog);
asio::ip::tcp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
diff --git a/extensions/standard-processors/tests/unit/ListenTcpTests.cpp b/extensions/standard-processors/tests/unit/ListenTcpTests.cpp
index 1daaf530f..42f80f4aa 100644
--- a/extensions/standard-processors/tests/unit/ListenTcpTests.cpp
+++ b/extensions/standard-processors/tests/unit/ListenTcpTests.cpp
@@ -38,11 +38,11 @@ void check_for_attributes(core::FlowFile& flow_file, uint16_t port) {
}
TEST_CASE("ListenTCP test multiple messages", "[ListenTCP][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenTCP>("ListenTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenTCP>("ListenTCP")};
const auto listen_tcp = controller.getProcessor<ListenTCP>();
LogTestController::getInstance().setTrace<ListenTCP>();
REQUIRE(listen_tcp->setProperty(ListenTCP::MaxBatchSize.name, "2"));
- auto port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ auto port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
asio::ip::tcp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -66,7 +66,7 @@ TEST_CASE("ListenTCP test multiple messages", "[ListenTCP][NetworkListenerProces
}
TEST_CASE("ListenTCP can be rescheduled", "[ListenTCP][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenTCP>("ListenTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenTCP>("ListenTCP")};
const auto listen_tcp = controller.getProcessor<ListenTCP>();
LogTestController::getInstance().setTrace<ListenTCP>();
REQUIRE(listen_tcp->setProperty(ListenTCP::Port.name, "0"));
@@ -78,12 +78,12 @@ TEST_CASE("ListenTCP can be rescheduled", "[ListenTCP][NetworkListenerProcessor]
}
TEST_CASE("ListenTCP max queue and max batch size test", "[ListenTCP][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenTCP>("ListenTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenTCP>("ListenTCP")};
const auto listen_tcp = controller.getProcessor<ListenTCP>();
LogTestController::getInstance().setTrace<ListenTCP>();
REQUIRE(listen_tcp->setProperty(ListenTCP::MaxBatchSize.name, "10"));
REQUIRE(listen_tcp->setProperty(ListenTCP::MaxQueueSize.name, "50"));
- auto port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ auto port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
asio::ip::tcp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -112,7 +112,7 @@ TEST_CASE("ListenTCP max queue and max batch size test", "[ListenTCP][NetworkLis
TEST_CASE("Test ListenTCP with SSL connection", "[ListenTCP][NetworkListenerProcessor]") {
uint16_t port = 0;
- SingleProcessorTestController controller{std::make_unique<ListenTCP>("ListenTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenTCP>("ListenTCP")};
const auto listen_tcp = controller.getProcessor<ListenTCP>();
auto ssl_context_service = controller.plan->addController("SSLContextService", "SSLContextService");
LogTestController::getInstance().setTrace<ListenTCP>();
@@ -130,7 +130,7 @@ TEST_CASE("Test ListenTCP with SSL connection", "[ListenTCP][NetworkListenerProc
SECTION("Without client certificate verification") {
SECTION("Client certificate not required, Client Auth set to NONE by default") {
ssl_context_service->enable();
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
SECTION("sending through IPv6", "[IPv6]") {
if (utils::isIPv6Disabled())
SKIP("IPv6 is disabled");
@@ -143,7 +143,7 @@ TEST_CASE("Test ListenTCP with SSL connection", "[ListenTCP][NetworkListenerProc
SECTION("Client certificate not required, but validated if provided") {
REQUIRE(controller.plan->setProperty(listen_tcp, ListenTCP::ClientAuth, "WANT"));
ssl_context_service->enable();
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
SECTION("sending through IPv6", "[IPv6]") {
if (utils::isIPv6Disabled())
SKIP("IPv6 is disabled");
@@ -164,7 +164,7 @@ TEST_CASE("Test ListenTCP with SSL connection", "[ListenTCP][NetworkListenerProc
SECTION("Client certificate required") {
REQUIRE(controller.plan->setProperty(listen_tcp, ListenTCP::ClientAuth, "REQUIRED"));
ssl_context_service->enable();
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
SECTION("sending through IPv6", "[IPv6]") {
if (utils::isIPv6Disabled())
SKIP("IPv6 is disabled");
@@ -177,7 +177,7 @@ TEST_CASE("Test ListenTCP with SSL connection", "[ListenTCP][NetworkListenerProc
SECTION("Client certificate not required but validated") {
REQUIRE(controller.plan->setProperty(listen_tcp, ListenTCP::ClientAuth, "WANT"));
ssl_context_service->enable();
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
SECTION("sending through IPv6", "[IPv6]") {
if (utils::isIPv6Disabled())
SKIP("IPv6 is disabled");
@@ -203,7 +203,7 @@ TEST_CASE("Test ListenTCP with SSL connection", "[ListenTCP][NetworkListenerProc
SECTION("Required certificate not provided") {
ssl_context_service->enable();
REQUIRE(controller.plan->setProperty(listen_tcp, ListenTCP::ClientAuth, "REQUIRED"));
- port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
SECTION("sending through IPv6", "[IPv6]") {
if (utils::isIPv6Disabled())
SKIP("IPv6 is disabled");
@@ -241,7 +241,7 @@ bool isSslMethodAvailable(asio::ssl::context::method method) {
} // namespace
TEST_CASE("Test ListenTCP SSL/TLS compatibility", "[ListenTCP][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenTCP>("ListenTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenTCP>("ListenTCP")};
const auto listen_tcp = controller.getProcessor<ListenTCP>();
auto ssl_context_service = controller.plan->addController("SSLContextService", "SSLContextService");
LogTestController::getInstance().setTrace<ListenTCP>();
@@ -255,7 +255,7 @@ TEST_CASE("Test ListenTCP SSL/TLS compatibility", "[ListenTCP][NetworkListenerPr
REQUIRE(controller.plan->setProperty(listen_tcp, ListenTCP::ClientAuth, "REQUIRED"));
ssl_context_service->enable();
- uint16_t port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ uint16_t port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
asio::ip::tcp::endpoint endpoint = asio::ip::tcp::endpoint(asio::ip::address_v4::loopback(), port);
minifi::utils::net::SslData ssl_data;
@@ -288,7 +288,7 @@ TEST_CASE("Test ListenTCP SSL/TLS compatibility", "[ListenTCP][NetworkListenerPr
}
TEST_CASE("Custom delimiter", "[ListenTCP][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenTCP>("ListenTCP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenTCP>("ListenTCP")};
const auto listen_tcp = controller.getProcessor<ListenTCP>();
LogTestController::getInstance().setTrace<ListenTCP>();
@@ -297,7 +297,7 @@ TEST_CASE("Custom delimiter", "[ListenTCP][NetworkListenerProcessor]") {
REQUIRE(listen_tcp->setProperty(ListenTCP::MessageDelimiter.name, delimiter));
REQUIRE(listen_tcp->setProperty(ListenTCP::ConsumeDelimiter.name, fmt::format("{}", consume_delimiter)));
- auto port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_tcp);
+ auto port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_tcp);
asio::ip::tcp::endpoint endpoint = asio::ip::tcp::endpoint(asio::ip::address_v4::loopback(), port);
diff --git a/extensions/standard-processors/tests/unit/ListenUDPTests.cpp b/extensions/standard-processors/tests/unit/ListenUDPTests.cpp
index 7c8781210..10144c63c 100644
--- a/extensions/standard-processors/tests/unit/ListenUDPTests.cpp
+++ b/extensions/standard-processors/tests/unit/ListenUDPTests.cpp
@@ -36,13 +36,13 @@ void check_for_attributes(core::FlowFile& flow_file, uint16_t port) {
}
TEST_CASE("ListenUDP test multiple messages", "[ListenUDP][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenUDP>("ListenUDP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenUDP>("ListenUDP")};
const auto listen_udp = controller.getProcessor<ListenUDP>();
LogTestController::getInstance().setTrace<ListenUDP>();
REQUIRE(listen_udp->setProperty(ListenUDP::MaxBatchSize.name, "2"));
- auto port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_udp);
+ auto port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_udp);
asio::ip::udp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
@@ -68,8 +68,8 @@ TEST_CASE("ListenUDP test multiple messages", "[ListenUDP][NetworkListenerProces
}
TEST_CASE("ListenUDP can be rescheduled", "[ListenUDP][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenUDP>("ListenUDP")};
- const auto listen_udp = controller.getProcessor<ListenUDP>();
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenUDP>("ListenUDP")};
+ const auto listen_udp = controller.getProcessor();
LogTestController::getInstance().setTrace<ListenUDP>();
REQUIRE(listen_udp->setProperty(ListenUDP::Port.name, "0"));
REQUIRE(listen_udp->setProperty(ListenUDP::MaxBatchSize.name, "100"));
@@ -80,12 +80,12 @@ TEST_CASE("ListenUDP can be rescheduled", "[ListenUDP][NetworkListenerProcessor]
}
TEST_CASE("ListenUDP max queue and max batch size test", "[ListenUDP][NetworkListenerProcessor]") {
- SingleProcessorTestController controller{std::make_unique<ListenUDP>("ListenUDP")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<ListenUDP>("ListenUDP")};
const auto listen_udp = controller.getProcessor<ListenUDP>();
REQUIRE(listen_udp->setProperty(ListenUDP::MaxBatchSize.name, "10"));
REQUIRE(listen_udp->setProperty(ListenUDP::MaxQueueSize.name, "50"));
- auto port = utils::scheduleProcessorOnRandomPort(controller.plan, *listen_udp);
+ auto port = utils::scheduleProcessorOnRandomPort(controller.plan, listen_udp);
asio::ip::udp::endpoint endpoint;
SECTION("sending through IPv6", "[IPv6]") {
diff --git a/extensions/standard-processors/tests/unit/LogAttributeTests.cpp b/extensions/standard-processors/tests/unit/LogAttributeTests.cpp
index d87d89c29..b97121b2c 100644
--- a/extensions/standard-processors/tests/unit/LogAttributeTests.cpp
+++ b/extensions/standard-processors/tests/unit/LogAttributeTests.cpp
@@ -29,7 +29,7 @@ using namespace std::literals::chrono_literals;
namespace org::apache::nifi::minifi::test {
TEST_CASE("LogAttribute logs payload", "[LogAttribute]") {
- SingleProcessorTestController controller{std::make_unique<LogAttribute>("log_attribute")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<LogAttribute>("log_attribute")};
const auto log_attribute = controller.getProcessor();
LogTestController::getInstance().setTrace<LogAttribute>();
@@ -54,7 +54,7 @@ TEST_CASE("LogAttribute logs payload", "[LogAttribute]") {
}
TEST_CASE("LogAttribute LogLevel and LogPrefix", "[LogAttribute]") {
- SingleProcessorTestController controller{std::make_unique<LogAttribute>("log_attribute")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<LogAttribute>("log_attribute")};
const auto log_attribute = controller.getProcessor();
LogTestController::getInstance().setTrace<LogAttribute>();
@@ -81,7 +81,7 @@ TEST_CASE("LogAttribute LogLevel and LogPrefix", "[LogAttribute]") {
}
TEST_CASE("LogAttribute filtering attributes", "[LogAttribute]") {
- SingleProcessorTestController controller{std::make_unique<LogAttribute>("log_attribute")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<LogAttribute>("log_attribute")};
const auto log_attribute = controller.getProcessor();
LogTestController::getInstance().setTrace<LogAttribute>();
@@ -128,7 +128,7 @@ TEST_CASE("LogAttribute filtering attributes", "[LogAttribute]") {
}
TEST_CASE("LogAttribute batch test", "[LogAttribute]") {
- SingleProcessorTestController controller{std::make_unique<LogAttribute>("log_attribute")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<LogAttribute>("log_attribute")};
const auto log_attribute = controller.getProcessor();
const auto [flow_files_to_log, expected_success_flow_files] = GENERATE(
diff --git a/extensions/standard-processors/tests/unit/ProcessGroupTestUtils.h b/extensions/standard-processors/tests/unit/ProcessGroupTestUtils.h
index 8b1d49885..b74d6b261 100644
--- a/extensions/standard-processors/tests/unit/ProcessGroupTestUtils.h
+++ b/extensions/standard-processors/tests/unit/ProcessGroupTestUtils.h
@@ -378,7 +378,7 @@ void verifyProcessGroup(core::ProcessGroup& group, const Group& pattern) {
const auto& input_ports = ProcessGroupTestAccessor::get_processors_(*rpg);
REQUIRE(input_ports.size() == expected.input_ports.size());
for (auto& expected_input_port : expected.input_ports) {
- auto input_port = dynamic_cast<minifi::RemoteProcessorGroupPort*>(findByName(input_ports, expected_input_port.name));
+ TypedProcessorWrapper<minifi::RemoteProcessorGroupPort> input_port = findByName(input_ports, expected_input_port.name);
REQUIRE(input_port);
REQUIRE(input_port->getName() == expected_input_port.name);
}
diff --git a/extensions/standard-processors/tests/unit/ProcessorTests.cpp b/extensions/standard-processors/tests/unit/ProcessorTests.cpp
index 8e8acb93f..9bd45174c 100644
--- a/extensions/standard-processors/tests/unit/ProcessorTests.cpp
+++ b/extensions/standard-processors/tests/unit/ProcessorTests.cpp
@@ -54,7 +54,7 @@
TEST_CASE("Test Creation of GetFile", "[getfileCreate]") {
TestController testController;
- auto processor = std::make_shared<minifi::processors::GetFile>("processorname");
+ auto processor = minifi::test::utils::make_processor<minifi::processors::GetFile>("processorname");
REQUIRE(processor->getName() == "processorname");
}
@@ -67,7 +67,7 @@ TEST_CASE("Test GetFileMultiple", "[getfileCreate3]") {
auto config = std::make_shared<minifi::ConfigureImpl>();
std::shared_ptr<core::ContentRepository> content_repo = std::make_shared<core::repository::VolatileContentRepository>();
content_repo->initialize(config);
- auto processor = std::make_shared<minifi::processors::GetFile>("getfileCreate2");
+ auto processor = minifi::test::utils::make_processor<minifi::processors::GetFile>("getfileCreate2");
processor->initialize();
std::shared_ptr<core::Repository> test_repo = std::make_shared<TestRepository>();
std::shared_ptr<TestRepository> repo = std::dynamic_pointer_cast<TestRepository>(test_repo);
@@ -147,7 +147,7 @@ TEST_CASE("Test GetFile Ignore", "[getfileCreate3]") {
TestController testController;
LogTestController::getInstance().setDebug<minifi::processors::GetFile>();
std::shared_ptr<core::ContentRepository> content_repo = std::make_shared<core::repository::VolatileContentRepository>();
- auto processor = std::make_shared<minifi::processors::GetFile>("getfileCreate2");
+ auto processor = minifi::test::utils::make_processor<minifi::processors::GetFile>("getfileCreate2");
processor->initialize();
std::shared_ptr<core::Repository> test_repo = std::make_shared<TestRepository>();
@@ -233,7 +233,7 @@ TEST_CASE("TestConnectionFull", "[ConnectionFull]") {
LogTestController::getInstance().setDebug<minifi::processors::GenerateFlowFile>();
std::shared_ptr<core::ContentRepository> content_repo = std::make_shared<core::repository::VolatileContentRepository>();
content_repo->initialize(std::make_shared<minifi::ConfigureImpl>());
- auto processor = std::make_shared<org::apache::nifi::minifi::processors::GenerateFlowFile>("GFF");
+ auto processor = minifi::test::utils::make_processor<org::apache::nifi::minifi::processors::GenerateFlowFile>("GFF");
processor->initialize();
REQUIRE(processor->setProperty(minifi::processors::GenerateFlowFile::BatchSize.name, "10"));
REQUIRE(processor->setProperty(minifi::processors::GenerateFlowFile::FileSize.name, "0"));
@@ -410,7 +410,7 @@ TEST_CASE("Test Find file", "[getfileCreate3]") {
LogTestController::getInstance().setDebug<minifi::provenance::ProvenanceReporter>();
std::shared_ptr<TestPlan> plan = testController.createPlan();
auto processor = plan->addProcessor("GetFile", "getfileCreate2");
- auto processorReport = plan->addProcessor(std::make_unique<minifi::core::reporting::SiteToSiteProvenanceReportingTask>(
+ TypedProcessorWrapper<minifi::core::reporting::SiteToSiteProvenanceReportingTask> processorReport = plan->addProcessor(minifi::test::utils::make_custom_processor<minifi::core::reporting::SiteToSiteProvenanceReportingTask>(
std::make_shared<minifi::ConfigureImpl>()), "reporter", core::Relationship("success", "description"), false);
auto dir = testController.createTempDirectory();
@@ -461,7 +461,7 @@ TEST_CASE("Test Find file", "[getfileCreate3]") {
throw std::runtime_error("Did not find record");
}
}
- auto taskReport = dynamic_cast<minifi::core::reporting::SiteToSiteProvenanceReportingTask*>(processorReport);
+ auto taskReport = &processorReport.get();
taskReport->setBatchSize(1);
std::vector<std::shared_ptr<core::SerializableComponent>> recordsReport;
recordsReport.push_back(std::make_shared<minifi::provenance::ProvenanceEventRecordImpl>());
@@ -483,9 +483,7 @@ TEST_CASE("Test Find file", "[getfileCreate3]") {
class TestProcessorNoContent : public minifi::core::ProcessorImpl {
public:
- explicit TestProcessorNoContent(std::string_view name, const utils::Identifier& uuid = {})
- : ProcessorImpl(name, uuid) {
- }
+ using ProcessorImpl::ProcessorImpl;
static constexpr const char* Description = "test resource";
static constexpr auto Properties = std::array<core::PropertyReference, 0>{};
@@ -544,7 +542,7 @@ void testRPGBypass(const std::string &host, const std::string &port, bool has_er
std::shared_ptr<core::Repository> test_repo = std::make_shared<TestRepository>();
std::shared_ptr<TestRepository> repo = std::dynamic_pointer_cast<TestRepository>(test_repo);
- auto rpg = std::make_shared<minifi::RemoteProcessorGroupPort>("rpg", "http://localhost:8989/nifi", configuration);
+ auto rpg = minifi::test::utils::make_custom_processor<minifi::RemoteProcessorGroupPort>("rpg", "http://localhost:8989/nifi", configuration, utils::IdGenerator::getIdGenerator()->generate());
rpg->initialize();
REQUIRE(rpg->setProperty(minifi::RemoteProcessorGroupPort::hostName.name, host));
REQUIRE(rpg->setProperty(minifi::RemoteProcessorGroupPort::port.name, port));
@@ -594,7 +592,7 @@ class ProcessorWithIncomingConnectionTest {
~ProcessorWithIncomingConnectionTest();
protected:
- std::shared_ptr<minifi::processors::LogAttribute> processor_;
+ std::shared_ptr<minifi::core::Processor> processor_;
std::shared_ptr<minifi::Connection> incoming_connection_;
std::shared_ptr<core::ProcessSession> session_;
};
@@ -606,7 +604,7 @@ ProcessorWithIncomingConnectionTest::ProcessorWithIncomingConnectionTest() {
const auto content_repo = std::make_shared<core::repository::VolatileContentRepository>();
content_repo->initialize(std::make_shared<minifi::ConfigureImpl>());
- processor_ = std::make_shared<minifi::processors::LogAttribute>("test_processor");
+ processor_ = minifi::test::utils::make_processor<minifi::processors::LogAttribute>("test_processor");
incoming_connection_ = std::make_shared<minifi::ConnectionImpl>(repo, content_repo, "incoming_connection");
incoming_connection_->addRelationship(core::Relationship{"success", ""});
incoming_connection_->setDestinationUUID(processor_->getUUID());
@@ -810,7 +808,7 @@ TEST_CASE("Test getProcessorType", "[getProcessorType]") {
}
TEST_CASE("IsYield and getYieldTime is consistent") {
- auto processor = TestProcessorNoContent("test_processor");
- processor.yield(1ms);
- REQUIRE(processor.isYield() == (processor.getYieldTime() != 0ms));
+ auto processor = minifi::test::utils::make_processor<TestProcessorNoContent>("test_processor");
+ processor->yield(1ms);
+ REQUIRE(processor->isYield() == (processor->getYieldTime() != 0ms));
}
diff --git a/extensions/standard-processors/tests/unit/PutFileTests.cpp b/extensions/standard-processors/tests/unit/PutFileTests.cpp
index 0bfc51135..d257780a0 100644
--- a/extensions/standard-processors/tests/unit/PutFileTests.cpp
+++ b/extensions/standard-processors/tests/unit/PutFileTests.cpp
@@ -38,7 +38,7 @@
TEST_CASE("Test Creation of PutFile", "[getfileCreate]") {
TestController testController;
- auto processor = std::make_shared<org::apache::nifi::minifi::processors::PutFile>("processorname");
+ auto processor = minifi::test::utils::make_processor<org::apache::nifi::minifi::processors::PutFile>("processorname");
REQUIRE(processor->getName() == "processorname");
}
diff --git a/extensions/standard-processors/tests/unit/PutTCPTests.cpp b/extensions/standard-processors/tests/unit/PutTCPTests.cpp
index ccac79638..8f9dd7528 100644
--- a/extensions/standard-processors/tests/unit/PutTCPTests.cpp
+++ b/extensions/standard-processors/tests/unit/PutTCPTests.cpp
@@ -98,8 +98,8 @@ utils::net::SslData createSslDataForServer() {
class PutTCPTestFixture {
public:
PutTCPTestFixture() :
- controller_(std::make_unique<PutTCP>("PutTCP")),
- put_tcp_(controller_.getProcessor<PutTCP>()) {
+ controller_(minifi::test::utils::make_processor<PutTCP>("PutTCP")),
+ put_tcp_(controller_.getProcessor()) {
LogTestController::getInstance().setTrace<PutTCP>();
LogTestController::getInstance().setInfo<core::ProcessSession>();
LogTestController::getInstance().setTrace<utils::net::Server>();
@@ -230,7 +230,7 @@ class PutTCPTestFixture {
}
test::SingleProcessorTestController controller_;
- PutTCP* put_tcp_;
+ minifi::core::Processor* put_tcp_;
class Server {
public:
diff --git a/extensions/standard-processors/tests/unit/PutUDPTests.cpp b/extensions/standard-processors/tests/unit/PutUDPTests.cpp
index 57974abac..dca54600d 100644
--- a/extensions/standard-processors/tests/unit/PutUDPTests.cpp
+++ b/extensions/standard-processors/tests/unit/PutUDPTests.cpp
@@ -27,6 +27,7 @@
#include "utils/net/UdpServer.h"
#include "utils/expected.h"
#include "utils/StringUtils.h"
+#include "unit/TestUtils.h"
using namespace std::literals::chrono_literals;
@@ -46,7 +47,7 @@ std::optional<utils::net::Message> tryDequeueWithTimeout(utils::net::UdpServer&
} // namespace
TEST_CASE("PutUDP", "[putudp]") {
- test::SingleProcessorTestController controller{std::make_unique<PutUDP>("PutUDP")};
+ test::SingleProcessorTestController controller{minifi::test::utils::make_processor<PutUDP>("PutUDP")};
const auto put_udp = controller.getProcessor();
LogTestController::getInstance().setTrace<PutUDP>();
diff --git a/extensions/standard-processors/tests/unit/ReplaceTextTests.cpp b/extensions/standard-processors/tests/unit/ReplaceTextTests.cpp
index 924b44722..72d0a55b1 100644
--- a/extensions/standard-processors/tests/unit/ReplaceTextTests.cpp
+++ b/extensions/standard-processors/tests/unit/ReplaceTextTests.cpp
@@ -27,7 +27,13 @@ struct ReplaceTextTestAccessor {
ReplaceText processor_;
ReplaceText::Parameters parameters_;
- ReplaceTextTestAccessor() : processor_{"replace_text"} {}
+ ReplaceTextTestAccessor()
+ : processor_{minifi::core::ProcessorMetadata{
+ .uuid = minifi::utils::IdGenerator::getIdGenerator()->generate(),
+ .name = "replace_text",
+ .logger = minifi::core::logging::LoggerFactory<ReplaceText>::getLogger()
+ }}
+ {}
void setEvaluationMode(EvaluationModeType evaluation_mode) { processor_.evaluation_mode_ = evaluation_mode; }
void setReplacementStrategy(ReplacementStrategyType replacement_strategy) { processor_.replacement_strategy_ = replacement_strategy; }
diff --git a/extensions/standard-processors/tests/unit/RouteTextTests.cpp b/extensions/standard-processors/tests/unit/RouteTextTests.cpp
index c51768bd8..918898224 100644
--- a/extensions/standard-processors/tests/unit/RouteTextTests.cpp
+++ b/extensions/standard-processors/tests/unit/RouteTextTests.cpp
@@ -20,6 +20,7 @@
#include "unit/TestBase.h"
#include "unit/Catch.h"
#include "processors/RouteText.h"
+#include "unit/TestUtils.h"
namespace processors = minifi::processors;
@@ -54,7 +55,7 @@ struct RouteTextController : public TestController {
RouteTextController() {
plan_ = createPlan();
- proc_ = plan_->addProcessor(std::make_unique<processors::RouteText>("RouteText"), "RouteText");
+ proc_ = plan_->addProcessor(minifi::test::utils::make_processor<processors::RouteText>("RouteText"), "RouteText");
input_ = plan_->addConnection(nullptr, {"success", ""}, proc_);
createOutput(processors::RouteText::Original);
createOutput(processors::RouteText::Unmatched);
diff --git a/extensions/standard-processors/tests/unit/SegmentContentTests.cpp b/extensions/standard-processors/tests/unit/SegmentContentTests.cpp
index bb4cc3158..841085896 100644
--- a/extensions/standard-processors/tests/unit/SegmentContentTests.cpp
+++ b/extensions/standard-processors/tests/unit/SegmentContentTests.cpp
@@ -26,6 +26,7 @@
#include "unit/SingleProcessorTestController.h"
#include "unit/TestBase.h"
#include "range/v3/algorithm/generate.hpp"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::processors::test {
@@ -56,7 +57,7 @@ std::vector<std::byte> createByteVector(Bytes... bytes) {
}
TEST_CASE("Invalid segmentSize tests") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SegmentContent>("SegmentContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SegmentContent>("SegmentContent")};
const auto segment_content = controller.getProcessor();
SECTION("foo") {
@@ -78,7 +79,7 @@ TEST_CASE("Invalid segmentSize tests") {
}
TEST_CASE("EmptyFlowFile") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SegmentContent>("SegmentContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SegmentContent>("SegmentContent")};
const auto segment_content = controller.getProcessor();
REQUIRE(segment_content->setProperty(SegmentContent::SegmentSize.name, "10 B"));
@@ -94,7 +95,7 @@ TEST_CASE("EmptyFlowFile") {
}
TEST_CASE("SegmentContent with different sized text input") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SegmentContent>("SegmentContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SegmentContent>("SegmentContent")};
const auto segment_content = controller.getProcessor();
auto [original_size, segment_size] = GENERATE(
@@ -131,7 +132,7 @@ TEST_CASE("SegmentContent with different sized text input") {
}
TEST_CASE("SegmentContent with different sized byte input") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SegmentContent>("SegmentContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SegmentContent>("SegmentContent")};
const auto segment_content = controller.getProcessor();
auto [original_size, segment_size] = GENERATE(
@@ -169,7 +170,7 @@ TEST_CASE("SegmentContent with different sized byte input") {
}
TEST_CASE("SimpleTest", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SegmentContent>("SegmentContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SegmentContent>("SegmentContent")};
const auto segment_content = controller.getProcessor();
REQUIRE(segment_content->setProperty(SegmentContent::SegmentSize.name, "4 B"));
@@ -214,7 +215,7 @@ TEST_CASE("SimpleTest", "[NiFi]") {
}
TEST_CASE("TransferSmall", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SegmentContent>("SegmentContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SegmentContent>("SegmentContent")};
const auto segment_content = controller.getProcessor();
REQUIRE(segment_content->setProperty(SegmentContent::SegmentSize.name, "4 KB"));
@@ -235,7 +236,7 @@ TEST_CASE("TransferSmall", "[NiFi]") {
}
TEST_CASE("ExpressionLanguageSupport", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SegmentContent>("SegmentContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SegmentContent>("SegmentContent")};
const auto segment_content = controller.getProcessor();
REQUIRE(segment_content->setProperty(SegmentContent::SegmentSize.name, "${segmentSize}"));
diff --git a/extensions/standard-processors/tests/unit/SplitContentTests.cpp b/extensions/standard-processors/tests/unit/SplitContentTests.cpp
index 712a638ef..b5ed5ce42 100644
--- a/extensions/standard-processors/tests/unit/SplitContentTests.cpp
+++ b/extensions/standard-processors/tests/unit/SplitContentTests.cpp
@@ -23,6 +23,7 @@
#include "unit/SingleProcessorTestController.h"
#include "unit/TestBase.h"
#include "utils/ConfigurationUtils.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::processors::test {
@@ -32,7 +33,7 @@ std::vector<std::byte> createByteVector(Bytes... bytes) {
}
TEST_CASE("WithoutByteSequence") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "true"));
@@ -42,7 +43,7 @@ TEST_CASE("WithoutByteSequence") {
}
TEST_CASE("EmptyFlowFile") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "ub"));
@@ -60,7 +61,7 @@ TEST_CASE("EmptyFlowFile") {
}
TEST_CASE("TextFormatLeadingPosition", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "ub"));
@@ -83,7 +84,7 @@ TEST_CASE("TextFormatLeadingPosition", "[NiFi]") {
}
TEST_CASE("TextFormatTrailingPosition", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
@@ -107,7 +108,7 @@ TEST_CASE("TextFormatTrailingPosition", "[NiFi]") {
}
TEST_CASE("TextFormatSplits", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
@@ -145,7 +146,7 @@ TEST_CASE("TextFormatSplits", "[NiFi]") {
}
TEST_CASE("SmallSplits", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "false"));
@@ -170,7 +171,7 @@ TEST_CASE("SmallSplits", "[NiFi]") {
}
TEST_CASE("WithSingleByteSplit", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "false"));
@@ -195,7 +196,7 @@ TEST_CASE("WithSingleByteSplit", "[NiFi]") {
}
TEST_CASE("WithLargerSplit", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "false"));
@@ -220,7 +221,7 @@ TEST_CASE("WithLargerSplit", "[NiFi]") {
}
TEST_CASE("KeepingSequence", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "true"));
@@ -245,7 +246,7 @@ TEST_CASE("KeepingSequence", "[NiFi]") {
}
TEST_CASE("EndsWithSequence", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "false"));
@@ -268,7 +269,7 @@ TEST_CASE("EndsWithSequence", "[NiFi]") {
}
TEST_CASE("EndsWithSequenceAndKeepSequence", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "true"));
@@ -291,7 +292,7 @@ TEST_CASE("EndsWithSequenceAndKeepSequence", "[NiFi]") {
}
TEST_CASE("StartsWithSequence", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "false"));
@@ -314,7 +315,7 @@ TEST_CASE("StartsWithSequence", "[NiFi]") {
}
TEST_CASE("StartsWithSequenceAndKeepTrailingSequence", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "true"));
@@ -340,7 +341,7 @@ TEST_CASE("StartsWithSequenceAndKeepTrailingSequence", "[NiFi]") {
}
TEST_CASE("StartsWithSequenceAndKeepLeadingSequence") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "true"));
@@ -362,7 +363,7 @@ TEST_CASE("StartsWithSequenceAndKeepLeadingSequence") {
}
TEST_CASE("StartsWithDoubleSequenceAndKeepLeadingSequence") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::KeepByteSequence.name, "true"));
@@ -388,7 +389,7 @@ TEST_CASE("StartsWithDoubleSequenceAndKeepLeadingSequence") {
}
TEST_CASE("NoSplitterInString", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
@@ -416,7 +417,7 @@ TEST_CASE("NoSplitterInString", "[NiFi]") {
}
TEST_CASE("ByteSequenceAtBufferTargetSize") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
static_assert(utils::configuration::DEFAULT_BUFFER_SIZE >= 10);
@@ -458,7 +459,7 @@ TEST_CASE("ByteSequenceAtBufferTargetSize") {
}
TEST_CASE("TrickyWithLeading", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "aab"));
@@ -479,7 +480,7 @@ TEST_CASE("TrickyWithLeading", "[NiFi]") {
}
TEST_CASE("TrickyWithTrailing", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "aab"));
@@ -500,7 +501,7 @@ TEST_CASE("TrickyWithTrailing", "[NiFi]") {
}
TEST_CASE("TrickierWithLeading", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "abcd"));
@@ -521,7 +522,7 @@ TEST_CASE("TrickierWithLeading", "[NiFi]") {
}
TEST_CASE("TrickierWithTrailing", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "abcd"));
@@ -542,7 +543,7 @@ TEST_CASE("TrickierWithTrailing", "[NiFi]") {
}
TEST_CASE("OnlyByteSequencesNoKeep", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "ab"));
@@ -558,7 +559,7 @@ TEST_CASE("OnlyByteSequencesNoKeep", "[NiFi]") {
}
TEST_CASE("OnlyByteSequencesTrailing", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "ab"));
@@ -578,7 +579,7 @@ TEST_CASE("OnlyByteSequencesTrailing", "[NiFi]") {
}
TEST_CASE("OnlyByteSequencesLeading", "[NiFi]") {
- minifi::test::SingleProcessorTestController controller{std::make_unique<SplitContent>("SplitContent")};
+ minifi::test::SingleProcessorTestController controller{minifi::test::utils::make_processor<SplitContent>("SplitContent")};
const auto split_content = controller.getProcessor();
REQUIRE(split_content->setProperty(SplitContent::ByteSequenceFormatProperty.name, std::string{magic_enum::enum_name(SplitContent::ByteSequenceFormat::Text)}));
REQUIRE(split_content->setProperty(SplitContent::ByteSequence.name, "ab"));
diff --git a/extensions/standard-processors/tests/unit/SplitRecordTests.cpp b/extensions/standard-processors/tests/unit/SplitRecordTests.cpp
index 16e58ba95..f0111f332 100644
--- a/extensions/standard-processors/tests/unit/SplitRecordTests.cpp
+++ b/extensions/standard-processors/tests/unit/SplitRecordTests.cpp
@@ -22,6 +22,7 @@
#include "unit/SingleProcessorTestController.h"
#include "utils/StringUtils.h"
#include "rapidjson/document.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::test {
@@ -56,7 +57,7 @@ class SplitRecordTestController : public TestController {
}
protected:
- SingleProcessorTestController controller_{std::make_unique<processors::SplitRecord>("SplitRecord")};
+ SingleProcessorTestController controller_{minifi::test::utils::make_processor<processors::SplitRecord>("SplitRecord")};
};
TEST_CASE_METHOD(SplitRecordTestController, "Invalid Records Per Split property", "[splitrecord]") {
diff --git a/extensions/standard-processors/tests/unit/SplitTextTests.cpp b/extensions/standard-processors/tests/unit/SplitTextTests.cpp
index 66f6adb8d..92a223249 100644
--- a/extensions/standard-processors/tests/unit/SplitTextTests.cpp
+++ b/extensions/standard-processors/tests/unit/SplitTextTests.cpp
@@ -23,6 +23,7 @@
#include "unit/SingleProcessorTestController.h"
#include "io/BufferStream.h"
#include "utils/ConfigurationUtils.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::test {
@@ -167,7 +168,7 @@ void verifySplitResults(const SingleProcessorTestController& controller, const P
}
void runSplitTextTest(const std::string& input, const std::vector<ExpectedSplitTextResult>& expected_results, const SplitTextProperties& properties) {
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::LineSplitCount.name, std::to_string(properties.line_split_count)));
if (properties.maximum_fragment_size) {
@@ -191,19 +192,19 @@ void runSplitTextTest(const std::string& input, const std::vector<ExpectedSplitT
}
TEST_CASE("Line Split Count property is required") {
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
REQUIRE_THROWS_WITH(controller.trigger("", {}), "Expected parsable uint64_t from \"SplitText::Line Split Count\", but got PropertyNotSet (Property Error:2)");
}
TEST_CASE("Line Split Count property can only be 0 if Maximum Fragment Size is set") {
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::LineSplitCount.name, "0"));
REQUIRE_THROWS_AS(controller.trigger("", {}), minifi::Exception);
}
TEST_CASE("Maximum Fragment Size cannot be set to 0") {
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::LineSplitCount.name, "0"));
REQUIRE(split_text->setProperty(processors::SplitText::MaximumFragmentSize.name, "0 B"));
@@ -211,7 +212,7 @@ TEST_CASE("Maximum Fragment Size cannot be set to 0") {
}
TEST_CASE("Header Line Marker Characters size cannot be equal or larger than split text buffer size") {
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::LineSplitCount.name, "1"));
std::string header_marker_character(BUFFER_SIZE, 'A');
@@ -221,7 +222,7 @@ TEST_CASE("Header Line Marker Characters size cannot be equal or larger than spl
TEST_CASE("SplitText only forwards empty flowfile") {
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::LineSplitCount.name, "1"));
const auto trigger_results = controller.trigger("", {{std::string(core::SpecialFlowAttribute::FILENAME), "a.foo"}});
@@ -406,7 +407,7 @@ TEST_CASE("Endlines are trimmed when Remove Trailing Newlines is set to true and
}
TEST_CASE("If flowfile is empty after trailing new lines are removed then flow file is not emitted") {
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
SECTION("Line split count 1") {
REQUIRE(split_text->setProperty(processors::SplitText::LineSplitCount.name, "1"));
@@ -602,7 +603,7 @@ TEST_CASE("If the header defined by the header line count is larger than the flo
SECTION("Header line count is one line shorter larger than the flow file") {
input = "header line 1\nheader line 2\nthis is a new line\n";
}
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::LineSplitCount.name, "1"));
REQUIRE(split_text->setProperty(processors::SplitText::HeaderLineCount.name, "4"));
@@ -616,7 +617,7 @@ TEST_CASE("If the header defined by the header line count is larger than the flo
TEST_CASE("If the header defined by the header line count is larger than the max fragment size then the processor should fail") {
std::string input;
input = "header line 1\nheader line 2\nthis is a new line\n";
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::MaximumFragmentSize.name, "20 B"));
REQUIRE(split_text->setProperty(processors::SplitText::HeaderLineCount.name, "2"));
@@ -630,7 +631,7 @@ TEST_CASE("If the header defined by the header line count is larger than the max
TEST_CASE("If header line count is the same as the flow file line count then no new flow file should be emitted") {
std::string input = "header line 1\nheader line 2\nthis is a new line\n";
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::LineSplitCount.name, "1"));
REQUIRE(split_text->setProperty(processors::SplitText::HeaderLineCount.name, "3"));
@@ -809,7 +810,7 @@ TEST_CASE("If a split fragment would only consist of new lines then only the tri
TEST_CASE("If the header defined by header marker characters is larger than the max fragments size then the processor should fail") {
std::string input;
input = "header line 1\nheader line 2\nthis is a new line\n";
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::MaximumFragmentSize.name, "20 B"));
REQUIRE(split_text->setProperty(processors::SplitText::HeaderLineMarkerCharacters.name, "hea"));
@@ -824,7 +825,7 @@ TEST_CASE("If the header defined by header marker characters is larger than the
TEST_CASE("If the header defined by header marker characters is the only content in the flow file then the processor should not emit new flow files") {
std::string input;
input = "header line 1\nheader line 2\n";
- SingleProcessorTestController controller{std::make_unique<processors::SplitText>("SplitText")};
+ SingleProcessorTestController controller{minifi::test::utils::make_processor<processors::SplitText>("SplitText")};
const auto split_text = controller.getProcessor();
REQUIRE(split_text->setProperty(processors::SplitText::MaximumFragmentSize.name, "40 B"));
REQUIRE(split_text->setProperty(processors::SplitText::HeaderLineMarkerCharacters.name, "hea"));
diff --git a/extensions/standard-processors/tests/unit/TailFileTests.cpp b/extensions/standard-processors/tests/unit/TailFileTests.cpp
index 843b553e6..bae6ac500 100644
--- a/extensions/standard-processors/tests/unit/TailFileTests.cpp
+++ b/extensions/standard-processors/tests/unit/TailFileTests.cpp
@@ -77,7 +77,7 @@ void appendTempFile(const std::filesystem::path& directory, const std::filesyste
TEST_CASE("TailFile reads the file until the first delimiter then picks up the second line if a delimiter is written between runs", "[simple]") {
// Create and write to the test file
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<minifi::processors::TailFile>("TailFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<minifi::processors::TailFile>("TailFile"));
auto dir = test_controller.createTempDirectory();
auto temp_file_path = dir / TMP_FILE;
@@ -745,7 +745,7 @@ TEST_CASE("TailFile onSchedule throws in Multiple mode if the Base Directory doe
}
TEST_CASE("TailFile finds and finishes the renamed file and continues with the new log file", "[rotation]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<minifi::processors::TailFile>("TailFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<minifi::processors::TailFile>("TailFile"));
constexpr char DELIM = ',';
constexpr size_t expected_pieces = std::ranges::count(NEWLINE_FILE, DELIM); // The last piece is left as considered unfinished
@@ -812,7 +812,7 @@ TEST_CASE("TailFile finds and finishes the renamed file and continues with the n
}
TEST_CASE("TailFile finds and finishes multiple rotated files and continues with the new log file", "[rotation]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<minifi::processors::TailFile>("TailFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<minifi::processors::TailFile>("TailFile"));
LogTestController::getInstance().setTrace<minifi::processors::TailFile>();
@@ -1407,7 +1407,7 @@ TEST_CASE("TailFile interprets the lookup frequency property correctly", "[multi
auto plan = testController.createPlan();
- auto tail_file = plan->addProcessor("TailFile", "Tail");
+ auto tail_file = plan->addProcessor<minifi::processors::TailFile>("Tail");
plan->setProperty(tail_file, minifi::processors::TailFile::TailMode, "Multiple file");
plan->setProperty(tail_file, minifi::processors::TailFile::BaseDirectory, directory.string());
plan->setProperty(tail_file, minifi::processors::TailFile::FileName, ".*\\.log");
@@ -1418,9 +1418,7 @@ TEST_CASE("TailFile interprets the lookup frequency property correctly", "[multi
SECTION("Lookup frequency not set => defaults to 10 minutes") {
TestController::runSession(plan, true);
- const auto tail_file_processor = dynamic_cast<minifi::processors::TailFile*>(tail_file);
- REQUIRE(tail_file_processor);
- REQUIRE(tail_file_processor->getLookupFrequency() == std::chrono::minutes{10});
+ REQUIRE(tail_file.get().getLookupFrequency() == std::chrono::minutes{10});
}
SECTION("Lookup frequency set to zero => new files are picked up immediately") {
@@ -1785,7 +1783,7 @@ TEST_CASE("TailFile can use an AttributeProviderService", "[AttributeProviderSer
TEST_CASE("TailFile honors batch size for maximum lines processed", "[batchSize]") {
LogTestController::getInstance().setTrace<minifi::processors::TailFile>();
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<minifi::processors::TailFile>("TailFile"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<minifi::processors::TailFile>("TailFile"));
auto tailfile = test_controller.getProcessor();
auto temp_file_path = test_controller.createTempDirectory() / TMP_FILE;
diff --git a/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp b/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp
index 72746076d..21137ad17 100644
--- a/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp
+++ b/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp
@@ -1588,7 +1588,7 @@ Parameter Context Name: my-context
std::unique_ptr<core::ProcessGroup> flow = yaml_config.getRootFromPayload(TEST_CONFIG_YAML);
REQUIRE(flow);
- auto* proc = dynamic_cast<core::ProcessorImpl*>(flow->findProcessorByName("DummyProcessor"));
+ auto* proc = flow->findProcessorByName("DummyProcessor");
REQUIRE(proc);
auto values = proc->getAllPropertyValues("Simple Property");
REQUIRE(values);
@@ -1639,7 +1639,7 @@ Parameter Context Name: my-context
REQUIRE(flow);
core::Property property("My Dynamic Property Sequence", "");
- auto* proc = dynamic_cast<core::ProcessorImpl*>(flow->findProcessorByName("DummyProcessor"));
+ auto* proc = flow->findProcessorByName("DummyProcessor");
REQUIRE(proc);
auto values = proc->getAllDynamicPropertyValues("My Dynamic Property Sequence");
REQUIRE(values);
@@ -1739,7 +1739,7 @@ Parameter Context Name: my-context
std::unique_ptr<core::ProcessGroup> flow = yaml_config.getRootFromPayload(TEST_CONFIG_YAML);
REQUIRE(flow);
- auto* proc = dynamic_cast<core::ProcessorImpl*>(flow->findProcessorByName("DummyProcessor"));
+ auto* proc = flow->findProcessorByName("DummyProcessor");
auto values = proc->getAllPropertyValues("Sensitive Property");
REQUIRE(values);
REQUIRE(values->size() == 2);
diff --git a/extensions/standard-processors/tests/unit/YamlConnectionParserTest.cpp b/extensions/standard-processors/tests/unit/YamlConnectionParserTest.cpp
index 12c92169e..73ab1f4e3 100644
--- a/extensions/standard-processors/tests/unit/YamlConnectionParserTest.cpp
+++ b/extensions/standard-processors/tests/unit/YamlConnectionParserTest.cpp
@@ -80,11 +80,19 @@ TEST_CASE("Connections components are parsed from yaml", "[YamlConfiguration]")
REQUIRE(231 == yaml_connection_parser.getSwapThreshold());
}
SECTION("Source and destination names and uuids are read") {
- const utils::Identifier expected_source_id = minifi::test::utils::generateUUID();
- const utils::Identifier expected_destination_id = minifi::test::utils::generateUUID();
+ const utils::Identifier expected_source_id = [&] {
+ auto proc = minifi::test::utils::make_processor<minifi::processors::TailFile>("TailFile_1");
+ auto uuid = proc->getUUID();
+ parent.addProcessor(std::move(proc));
+ return uuid;
+ }();
+ const utils::Identifier expected_destination_id = [&] {
+ auto proc = minifi::test::utils::make_processor<minifi::processors::TailFile>("TailFile_2");
+ auto uuid = proc->getUUID();
+ parent.addProcessor(std::move(proc));
+ return uuid;
+ }();
std::string serialized_yaml;
- parent.addProcessor(std::make_unique<minifi::processors::TailFile>("TailFile_1", expected_source_id));
- parent.addProcessor(std::make_unique<minifi::processors::TailFile>("TailFile_2", expected_destination_id));
SECTION("Directly from configuration") {
serialized_yaml = std::string {
"source id: " + expected_source_id.to_string() + "\n"
diff --git a/extensions/test-processors/KamikazeProcessor.h b/extensions/test-processors/KamikazeProcessor.h
index 996766bff..d2a58ae9d 100644
--- a/extensions/test-processors/KamikazeProcessor.h
+++ b/extensions/test-processors/KamikazeProcessor.h
@@ -39,10 +39,7 @@ class KamikazeProcessor : public core::ProcessorImpl {
EXTENSIONAPI static const std::string OnTriggerLogStr;
EXTENSIONAPI static const std::string OnUnScheduleLogStr;
- explicit KamikazeProcessor(const std::string_view name, const utils::Identifier& uuid = utils::Identifier())
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<KamikazeProcessor>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
EXTENSIONAPI static constexpr const char* Description = "This processor can throw exceptions in onTrigger and onSchedule calls based on configuration. Only for testing purposes.";
diff --git a/extensions/test-processors/LogOnDestructionProcessor.h b/extensions/test-processors/LogOnDestructionProcessor.h
index 2f808d815..1937dc9d6 100644
--- a/extensions/test-processors/LogOnDestructionProcessor.h
+++ b/extensions/test-processors/LogOnDestructionProcessor.h
@@ -29,10 +29,7 @@ namespace org::apache::nifi::minifi::processors {
class LogOnDestructionProcessor : public core::ProcessorImpl {
public:
- explicit LogOnDestructionProcessor(const std::string_view name, const utils::Identifier& uuid = utils::Identifier())
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<LogOnDestructionProcessor>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
~LogOnDestructionProcessor() override {
logger_->log_info("LogOnDestructionProcessor is being destructed");
diff --git a/libminifi/include/ForwardingNode.h b/libminifi/include/ForwardingNode.h
index ffa8e7713..914d62e43 100644
--- a/libminifi/include/ForwardingNode.h
+++ b/libminifi/include/ForwardingNode.h
@@ -29,8 +29,7 @@ namespace org::apache::nifi::minifi {
class ForwardingNode : public core::ProcessorImpl {
public:
- ForwardingNode(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger) : ProcessorImpl(name, uuid), logger_(std::move(logger)) {}
- ForwardingNode(std::string_view name, std::shared_ptr<core::logging::Logger> logger) : ProcessorImpl(name), logger_(std::move(logger)) {}
+ using ProcessorImpl::ProcessorImpl;
MINIFIAPI static constexpr auto Properties = std::array<core::PropertyReference, 0>{};
MINIFIAPI static constexpr auto Success = core::RelationshipDefinition{"success", "FlowFiles are routed to success relationship"};
@@ -42,9 +41,6 @@ class ForwardingNode : public core::ProcessorImpl {
void initialize() override;
void onTrigger(core::ProcessContext& context, core::ProcessSession& session) override;
-
- private:
- std::shared_ptr<core::logging::Logger> logger_;
};
} // namespace org::apache::nifi::minifi
diff --git a/libminifi/include/Funnel.h b/libminifi/include/Funnel.h
index b6b3a7363..f5a37b1df 100644
--- a/libminifi/include/Funnel.h
+++ b/libminifi/include/Funnel.h
@@ -26,8 +26,7 @@ namespace org::apache::nifi::minifi {
class Funnel final : public ForwardingNode {
public:
- Funnel(std::string_view name, const utils::Identifier& uuid) : ForwardingNode(name, uuid, core::logging::LoggerFactory<Funnel>::getLogger(uuid)) {}
- explicit Funnel(std::string_view name) : ForwardingNode(name, core::logging::LoggerFactory<Funnel>::getLogger()) {}
+ Funnel(std::string_view name, const utils::Identifier& uuid) : ForwardingNode(core::ProcessorMetadata{.uuid = uuid, .name = std::string{name}, .logger = core::logging::LoggerFactory<Funnel>::getLogger(uuid)}) {}
MINIFIAPI static constexpr core::annotation::Input InputRequirement = core::annotation::Input::INPUT_REQUIRED;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
diff --git a/libminifi/include/Port.h b/libminifi/include/Port.h
index c4186de2b..201d4faee 100644
--- a/libminifi/include/Port.h
+++ b/libminifi/include/Port.h
@@ -21,6 +21,7 @@
#include <utility>
#include "ForwardingNode.h"
+#include "core/ProcessorProxy.h"
namespace org::apache::nifi::minifi {
@@ -29,10 +30,14 @@ enum class PortType {
OUTPUT
};
-class Port final : public ForwardingNode {
+class Port : public virtual core::Processor {
public:
- Port(std::string_view name, const utils::Identifier& uuid, PortType port_type) : ForwardingNode(name, uuid, core::logging::LoggerFactory<Port>::getLogger(uuid)), port_type_(port_type) {}
- Port(std::string_view name, PortType port_type) : ForwardingNode(name, core::logging::LoggerFactory<Port>::getLogger()), port_type_(port_type) {}
+ virtual PortType getPortType() const = 0;
+};
+
+class PortImpl final : public ForwardingNode {
+ public:
+ PortImpl(std::string_view name, const utils::Identifier& uuid, PortType port_type) : ForwardingNode({.uuid = uuid, .name = std::string{name}, .logger = core::logging::LoggerFactory<Port>::getLogger(uuid)}), port_type_(port_type) {}
PortType getPortType() const {
return port_type_;
}
@@ -44,4 +49,15 @@ class Port final : public ForwardingNode {
PortType port_type_;
};
+class PortProxy : public virtual Port, public core::ProcessorProxy {
+ public:
+ PortProxy(std::string_view name, const utils::Identifier& uuid, std::unique_ptr<PortImpl> impl): ProcessorProxy(name, uuid, std::move(impl)) {}
+
+ PortType getPortType() const override {
+ auto* port_impl = dynamic_cast<const PortImpl*>(impl_.get());
+ gsl_Assert(port_impl);
+ return port_impl->getPortType();
+ }
+};
+
} // namespace org::apache::nifi::minifi
diff --git a/libminifi/include/RemoteProcessorGroupPort.h b/libminifi/include/RemoteProcessorGroupPort.h
index 0f19721ec..7374aad43 100644
--- a/libminifi/include/RemoteProcessorGroupPort.h
+++ b/libminifi/include/RemoteProcessorGroupPort.h
@@ -77,15 +77,14 @@ struct RPG {
class RemoteProcessorGroupPort : public core::ProcessorImpl {
public:
- RemoteProcessorGroupPort(std::string_view name, std::string url, std::shared_ptr<Configure> configure, const utils::Identifier &uuid = {})
- : core::ProcessorImpl(name, uuid),
+ RemoteProcessorGroupPort(std::string_view name, std::string url, std::shared_ptr<Configure> configure, const utils::Identifier &uuid, std::shared_ptr<core::logging::Logger> logger = nullptr)
+ : core::ProcessorImpl({.uuid = uuid, .name = std::string{name}, .logger = logger ? logger : core::logging::LoggerFactory<RemoteProcessorGroupPort>::getLogger(uuid)}),
configure_(std::move(configure)),
direction_(sitetosite::SEND),
transmitting_(false),
timeout_(0),
bypass_rest_api_(false),
- ssl_service(nullptr),
- logger_(core::logging::LoggerFactory<RemoteProcessorGroupPort>::getLogger(uuid)) {
+ ssl_service(nullptr) {
client_type_ = sitetosite::CLIENT_TYPE::RAW;
protocol_uuid_ = uuid;
site2site_secure_ = false;
@@ -251,7 +250,6 @@ class RemoteProcessorGroupPort : public core::ProcessorImpl {
std::shared_ptr<controllers::SSLContextServiceInterface> ssl_service;
private:
- std::shared_ptr<core::logging::Logger> logger_;
static const char* RPG_SSL_CONTEXT_SERVICE_NAME;
};
diff --git a/libminifi/include/core/Processor.h b/libminifi/include/core/Processor.h
index 8d4904827..285b16d2d 100644
--- a/libminifi/include/core/Processor.h
+++ b/libminifi/include/core/Processor.h
@@ -40,6 +40,7 @@ class Connection;
namespace core {
+class ProcessorApi;
class ProcessContext;
class ProcessSession;
class ProcessSessionFactory;
@@ -84,6 +85,17 @@ class Processor : public virtual Connectable, public virtual ConfigurableCompone
virtual void validateAnnotations() const = 0;
virtual annotation::Input getInputRequirement() const = 0;
virtual gsl::not_null<std::shared_ptr<ProcessorMetrics>> getMetrics() const = 0;
+ virtual std::string getProcessGroupUUIDStr() const = 0;
+ virtual void setProcessGroupUUIDStr(const std::string &uuid) = 0;
+
+ virtual ProcessorApi& getImpl() const = 0;
+
+ template<typename T>
+ T& getImpl() const {
+ auto* res = dynamic_cast<T*>(&getImpl());
+ gsl_Assert(res);
+ return *res;
+ }
virtual void updateReachability(const std::lock_guard<std::mutex>& graph_lock, bool force = false) = 0;
virtual const std::unordered_map<Connection*, std::unordered_set<Processor*>>& reachable_processors() const = 0;
diff --git a/libminifi/include/core/ProcessorProxy.h b/libminifi/include/core/ProcessorProxy.h
index 5ea8931ce..0ad48bb3e 100644
--- a/libminifi/include/core/ProcessorProxy.h
+++ b/libminifi/include/core/ProcessorProxy.h
@@ -152,6 +152,14 @@ class ProcessorProxy : public virtual Processor, public ConnectableImpl, public
active_tasks_ = 0;
}
+ std::string getProcessGroupUUIDStr() const override {
+ return process_group_uuid_;
+ }
+
+ void setProcessGroupUUIDStr(const std::string &uuid) override {
+ process_group_uuid_ = uuid;
+ }
+
void yield() override;
void yield(std::chrono::steady_clock::duration delta_time) override;
@@ -216,6 +224,15 @@ class ProcessorProxy : public virtual Processor, public ConnectableImpl, public
return impl_->getMetrics();
}
+ ProcessorApi& getImpl() const override {
+ gsl_Assert(impl_);
+ return *impl_;
+ }
+
+ void restore(const std::shared_ptr<FlowFile>& file) override {
+ impl_->restore(file);
+ }
+
static constexpr auto DynamicProperties = std::array<DynamicProperty, 0>{};
static constexpr auto OutputAttributes = std::array<OutputAttributeReference, 0>{};
@@ -254,6 +271,9 @@ class ProcessorProxy : public virtual Processor, public ConnectableImpl, public
// an outgoing connection allows us to reach these nodes
std::unordered_map<Connection*, std::unordered_set<Processor*>> reachable_processors_;
+ std::string process_group_uuid_;
+
+ protected:
std::unique_ptr<ProcessorApi> impl_;
};
diff --git a/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h b/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
index 50f5311a2..f79eb9eb7 100644
--- a/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
+++ b/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
@@ -35,8 +35,7 @@ namespace org::apache::nifi::minifi::core::reporting {
class SiteToSiteProvenanceReportingTask : public minifi::RemoteProcessorGroupPort {
public:
explicit SiteToSiteProvenanceReportingTask(std::shared_ptr<Configure> configure)
- : minifi::RemoteProcessorGroupPort(ReportTaskName, "", std::move(configure)),
- logger_(logging::LoggerFactory<SiteToSiteProvenanceReportingTask>::getLogger()) {
+ : minifi::RemoteProcessorGroupPort(ReportTaskName, "", std::move(configure), utils::IdGenerator::getIdGenerator()->generate(), logging::LoggerFactory<SiteToSiteProvenanceReportingTask>::getLogger()) {
this->setTriggerWhenEmpty(true);
batch_size_ = 100;
}
@@ -71,8 +70,6 @@ class SiteToSiteProvenanceReportingTask : public minifi::RemoteProcessorGroupPor
private:
int batch_size_;
-
- std::shared_ptr<logging::Logger> logger_;
};
// SiteToSiteProvenanceReportingTask
diff --git a/libminifi/src/core/ConfigurableComponentImpl.cpp b/libminifi/src/core/ConfigurableComponentImpl.cpp
new file mode 100644
index 000000000..908c6a0e8
--- /dev/null
+++ b/libminifi/src/core/ConfigurableComponentImpl.cpp
@@ -0,0 +1,149 @@
+///**
+// *
+// * Licensed to the Apache Software Foundation (ASF) under one or more
+// * contributor license agreements. See the NOTICE file distributed with
+// * this work for additional information regarding copyright ownership.
+// * The ASF licenses this file to You under the Apache License, Version 2.0
+// * (the "License"); you may not use this file except in compliance with
+// * the License. You may obtain a copy of the License at
+// *
+// * http://www.apache.org/licenses/LICENSE-2.0
+// *
+// * Unless required by applicable law or agreed to in writing, software
+// * distributed under the License is distributed on an "AS IS" BASIS,
+// * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// * See the License for the specific language governing permissions and
+// * limitations under the License.
+// */
+//
+//#include "core/ConfigurableComponentImpl.h"
+//
+//#include "minifi-cpp/core/Property.h"
+//#include "minifi-cpp/utils/PropertyErrors.h"
+//
+//namespace org::apache::nifi::minifi::core {
+//void ConfigurableComponentImpl::setSupportedProperties(std::span<const core::PropertyReference> properties) {
+// if (!canEdit()) { return; }
+//
+// std::lock_guard<std::mutex> lock(configuration_mutex_);
+//
+// supported_properties_.clear();
+// for (const auto& item: properties) { supported_properties_.emplace(item.name, item); }
+//}
+//
+//nonstd::expected<std::string, std::error_code> ConfigurableComponentImpl::getProperty(const std::string_view name) const {
+// const std::lock_guard<std::mutex> lock(configuration_mutex_);
+// const auto it = supported_properties_.find(name);
+// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
+// const Property& prop = it->second;
+// return prop.getValue() | utils::transform([](const std::string_view value_view) -> std::string { return std::string{value_view}; });
+//}
+//
+//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::setProperty(const std::string_view name, std::string value) {
+// const std::lock_guard<std::mutex> lock(configuration_mutex_);
+// const auto it = supported_properties_.find(name);
+// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
+// Property& prop = it->second;
+//
+// return prop.setValue(std::move(value));
+//}
+//
+//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::clearProperty(const std::string_view name) {
+// const std::lock_guard<std::mutex> lock(configuration_mutex_);
+// const auto it = supported_properties_.find(name);
+// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
+// Property& prop = it->second;
+//
+// prop.clearValues();
+// return {};
+//}
+//
+//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::appendProperty(const std::string_view name, std::string value) {
+// const std::lock_guard<std::mutex> lock(configuration_mutex_);
+// const auto it = supported_properties_.find(name);
+// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
+// Property& prop = it->second;
+//
+// return prop.appendValue(std::move(value));
+//}
+//
+//nonstd::expected<std::string, std::error_code> ConfigurableComponentImpl::getDynamicProperty(const std::string_view name) const {
+// const std::lock_guard<std::mutex> lock(configuration_mutex_);
+// if (!supportsDynamicProperties()) {
+// return nonstd::make_unexpected(PropertyErrorCode::DynamicPropertiesNotSupported);
+// }
+// const auto it = dynamic_properties_.find(name);
+// if (it == dynamic_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::PropertyNotSet); }
+// const Property& prop = it->second;
+// return prop.getValue() | utils::transform([](const std::string_view value_view) -> std::string { return std::string{value_view}; });
+//}
+//
+//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::setDynamicProperty(std::string name, std::string value) {
+// const std::lock_guard<std::mutex> lock(configuration_mutex_);
+// if (!supportsDynamicProperties()) {
+// return nonstd::make_unexpected(PropertyErrorCode::DynamicPropertiesNotSupported);
+// }
+// Property& prop = dynamic_properties_[std::move(name)];
+// return prop.setValue(std::move(value));
+//}
+//
+//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::appendDynamicProperty(const std::string_view name, std::string value) {
+// const std::lock_guard<std::mutex> lock(configuration_mutex_);
+// if (!supportsDynamicProperties()) {
+// return nonstd::make_unexpected(PropertyErrorCode::DynamicPropertiesNotSupported);
+// }
+// Property& prop = dynamic_properties_[std::string{name}];
+// return prop.appendValue(std::move(value));
+//}
+//
+//std::vector<std::string> ConfigurableComponentImpl::getDynamicPropertyKeys() const {
+// std::lock_guard<std::mutex> lock(configuration_mutex_);
+//
+// return dynamic_properties_ | ranges::views::transform([](const auto& kv) { return kv.first; }) | ranges::to<std::vector>();
+//}
+//
+//std::map<std::string, std::string> ConfigurableComponentImpl::getDynamicProperties() const {
+// std::lock_guard<std::mutex> lock(configuration_mutex_);
+// std::map<std::string, std::string> result;
+// for(const auto& [key,value]: dynamic_properties_) {
+// result[key] = value.getValue().value_or("");
+// }
+// return result;
+//}
+//
+//nonstd::expected<PropertyReference, std::error_code> ConfigurableComponentImpl::getPropertyReference(const std::string_view name) const {
+// const auto it = supported_properties_.find(name);
+// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
+// const Property& prop = it->second;
+// return prop.getReference();
+//}
+//
+//[[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> ConfigurableComponentImpl::getAllPropertyValues(const std::string_view name) const {
+// std::lock_guard<std::mutex> lock(configuration_mutex_);
+//
+// const auto it = supported_properties_.find(name);
+// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
+// const Property& prop = it->second;
+// return prop.getAllValues() | utils::transform([](const auto& values) -> std::vector<std::string> { return std::vector<std::string>{values.begin(), values.end()}; });
+//}
+//
+//[[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> ConfigurableComponentImpl::getAllDynamicPropertyValues(const std::string_view name) const {
+// std::lock_guard<std::mutex> lock(configuration_mutex_);
+//
+// const auto it = dynamic_properties_.find(name);
+// if (it == dynamic_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::PropertyNotSet); }
+// const Property& prop = it->second;
+// return prop.getAllValues() | utils::transform([](const auto& values) -> std::vector<std::string> { return std::vector<std::string>{values.begin(), values.end()}; });
+//}
+//
+//[[nodiscard]] std::map<std::string, Property> ConfigurableComponentImpl::getSupportedProperties() const {
+// std::lock_guard<std::mutex> lock(configuration_mutex_);
+// std::map<std::string, Property> supported_properties;
+// for (const auto& [name, prop]: supported_properties_) {
+// supported_properties.emplace(name, prop);
+// }
+// return supported_properties;
+//
+//}
+//
+//} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/src/core/Connectable.cpp b/libminifi/src/core/Connectable.cpp
new file mode 100644
index 000000000..5fe3f9f07
--- /dev/null
+++ b/libminifi/src/core/Connectable.cpp
@@ -0,0 +1,173 @@
+///**
+// *
+// * Licensed to the Apache Software Foundation (ASF) under one or more
+// * contributor license agreements. See the NOTICE file distributed with
+// * this work for additional information regarding copyright ownership.
+// * The ASF licenses this file to You under the Apache License, Version 2.0
+// * (the "License"); you may not use this file except in compliance with
+// * the License. You may obtain a copy of the License at
+// *
+// * http://www.apache.org/licenses/LICENSE-2.0
+// *
+// * Unless required by applicable law or agreed to in writing, software
+// * distributed under the License is distributed on an "AS IS" BASIS,
+// * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// * See the License for the specific language governing permissions and
+// * limitations under the License.
+// */
+//#include "core/ConnectableImpl.h"
+//#include <utility>
+//#include <memory>
+//#include <string>
+//#include <set>
+//#include "core/logging/LoggerFactory.h"
+//
+//namespace org::apache::nifi::minifi::core {
+//
+//ConnectableImpl::ConnectableImpl(const std::string_view name, const utils::Identifier &uuid)
+// : CoreComponentImpl(name, uuid),
+// max_concurrent_tasks_(1),
+// connectable_version_(nullptr),
+// logger_(logging::LoggerFactory<Connectable>::getLogger(uuid_)) {
+//}
+//
+//ConnectableImpl::ConnectableImpl(const std::string_view name)
+// : CoreComponentImpl(name),
+// max_concurrent_tasks_(1),
+// connectable_version_(nullptr),
+// logger_(logging::LoggerFactory<Connectable>::getLogger(uuid_)) {
+//}
+//
+//ConnectableImpl::~ConnectableImpl() = default;
+//
+//void ConnectableImpl::setSupportedRelationships(std::span<const core::RelationshipDefinition> relationships) {
+// if (isRunning()) {
+// logger_->log_warn("Cannot set processor supported relationship while the process {} is running", name_);
+// return;
+// }
+//
+// std::lock_guard<std::mutex> lock(relationship_mutex_);
+//
+// relationships_.clear();
+// for (const auto& item : relationships) {
+// relationships_.emplace(item.name, item);
+// logger_->log_debug("Processor {} supported relationship name {}", name_, item.name);
+// }
+//}
+//
+//std::vector<Relationship> ConnectableImpl::getSupportedRelationships() const {
+// std::vector<Relationship> relationships;
+// relationships.reserve(relationships_.size());
+// for (auto const &item : relationships_) {
+// relationships.push_back(item.second);
+// }
+// return relationships;
+//}
+//
+//bool ConnectableImpl::isSupportedRelationship(const core::Relationship &relationship) {
+// // if we are running we do not need a lock since the function to change relationships_ ( setSupportedRelationships)
+// // cannot be executed while we are running
+// const bool isConnectableRunning = isRunning();
+//
+// const auto conditionalLock = isConnectableRunning ? std::unique_lock<std::mutex>() : std::unique_lock<std::mutex>(relationship_mutex_);
+//
+// return relationships_.contains(relationship.getName());
+//}
+//
+//void ConnectableImpl::addAutoTerminatedRelationship(const core::Relationship& relationship) {
+// if (isRunning()) {
+// logger_->log_warn("Can not add processor auto terminated relationship while the process {} is running", name_);
+// return;
+// }
+//
+// std::lock_guard<std::mutex> lock(relationship_mutex_);
+//
+// auto_terminated_relationships_[relationship.getName()] = relationship;
+//}
+//
+//void ConnectableImpl::setAutoTerminatedRelationships(std::span<const core::Relationship> relationships) {
+// if (isRunning()) {
+// logger_->log_warn("Can not set processor auto terminated relationship while the process {} is running", name_);
+// return;
+// }
+//
+// std::lock_guard<std::mutex> lock(relationship_mutex_);
+//
+// auto_terminated_relationships_.clear();
+// for (const auto& item : relationships) {
+// auto_terminated_relationships_[item.getName()] = item;
+// logger_->log_debug("Processor {} auto terminated relationship name {}", name_, item.getName());
+// }
+//}
+//
+//bool ConnectableImpl::isAutoTerminated(const core::Relationship &relationship) {
+// // if we are running we do not need a lock since the function to change relationships_ ( setSupportedRelationships)
+// // cannot be executed while we are running
+// const bool isConnectableRunning = isRunning();
+//
+// const auto conditionalLock = isConnectableRunning ? std::unique_lock<std::mutex>() : std::unique_lock<std::mutex>(relationship_mutex_);
+//
+// return auto_terminated_relationships_.contains(relationship.getName());
+//}
+//
+//void ConnectableImpl::waitForWork(std::chrono::milliseconds timeout) {
+// has_work_.store(isWorkAvailable());
+//
+// if (!has_work_.load()) {
+// std::unique_lock<std::mutex> lock(work_available_mutex_);
+// work_condition_.wait_for(lock, timeout, [&] {return has_work_.load();});
+// }
+//}
+//
+//void ConnectableImpl::notifyWork() {
+// // Do nothing if we are not event-driven
+// if (strategy_ != EVENT_DRIVEN) {
+// return;
+// }
+//
+// {
+// has_work_.store(isWorkAvailable());
+//
+// if (has_work_.load()) {
+// work_condition_.notify_one();
+// }
+// }
+//}
+//
+//std::set<Connectable*> ConnectableImpl::getOutGoingConnections(const std::string &relationship) {
+// const auto it = outgoing_connections_.find(relationship);
+// if (it != outgoing_connections_.end()) {
+// return it->second;
+// } else if (relationship == "__self__") {
+// return {this};
+// } else {
+// return {};
+// }
+//}
+//
+//Connectable* ConnectableImpl::getNextIncomingConnection() {
+// std::lock_guard<std::mutex> lock(relationship_mutex_);
+// return getNextIncomingConnectionImpl(lock);
+//}
+//
+//Connectable* ConnectableImpl::getNextIncomingConnectionImpl(const std::lock_guard<std::mutex>& /*relatioship_mutex_lock*/) {
+// if (incoming_connections_.empty())
+// return nullptr;
+//
+// if (incoming_connections_Iter == incoming_connections_.end())
+// incoming_connections_Iter = incoming_connections_.begin();
+//
+// auto ret = *incoming_connections_Iter;
+// incoming_connections_Iter++;
+//
+// if (incoming_connections_Iter == incoming_connections_.end())
+// incoming_connections_Iter = incoming_connections_.begin();
+//
+// return ret;
+//}
+//
+//Connectable* ConnectableImpl::pickIncomingConnection() {
+// return getNextIncomingConnection();
+//}
+//
+//} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/src/core/FlowConfiguration.cpp b/libminifi/src/core/FlowConfiguration.cpp
index 17285480c..8c118a635 100644
--- a/libminifi/src/core/FlowConfiguration.cpp
+++ b/libminifi/src/core/FlowConfiguration.cpp
@@ -91,9 +91,7 @@ std::unique_ptr<core::Processor> FlowConfiguration::createProcessor(const std::s
}
std::unique_ptr<core::reporting::SiteToSiteProvenanceReportingTask> FlowConfiguration::createProvenanceReportTask() {
- auto processor = std::make_unique<org::apache::nifi::minifi::core::reporting::SiteToSiteProvenanceReportingTask>(this->configuration_);
- processor->initialize();
- return processor;
+ return std::make_unique<org::apache::nifi::minifi::core::reporting::SiteToSiteProvenanceReportingTask>(this->configuration_);
}
std::unique_ptr<core::ProcessGroup> FlowConfiguration::updateFromPayload(const std::string& url, const std::string& yamlConfigPayload, const std::optional<std::string>& flow_id) {
diff --git a/libminifi/src/core/ProcessContext.cpp b/libminifi/src/core/ProcessContext.cpp
index 63bb8bc8b..39e993108 100644
--- a/libminifi/src/core/ProcessContext.cpp
+++ b/libminifi/src/core/ProcessContext.cpp
@@ -51,6 +51,7 @@ ProcessContextImpl::ProcessContextImpl(
initialized_(false) {
repo_ = repo;
state_storage_ = getStateStorage(logger_, controller_service_provider_, nullptr);
+ info_ = std::make_unique<StandardProcessorInfo>(processor);
}
ProcessContextImpl::ProcessContextImpl(
@@ -71,6 +72,11 @@ ProcessContextImpl::ProcessContextImpl(
info_ = std::make_unique<StandardProcessorInfo>(processor);
}
+bool ProcessContextImpl::hasNonEmptyProperty(std::string_view name) const {
+ auto val = getProcessor().getProperty(name);
+ return val && !val->empty();
+}
+
std::vector<std::string> ProcessContextImpl::getDynamicPropertyKeys() const { return processor_.getDynamicPropertyKeys(); }
std::map<std::string, std::string> ProcessContextImpl::getDynamicProperties(const FlowFile*) const { return processor_.getDynamicProperties(); }
@@ -97,14 +103,34 @@ nonstd::expected<std::string, std::error_code> ProcessContextImpl::getDynamicPro
return getProcessor().getDynamicProperty(name);
}
+nonstd::expected<std::string, std::error_code> ProcessContextImpl::getRawProperty(const std::string_view name) const {
+ return getProcessor().getProperty(name);
+}
+
nonstd::expected<void, std::error_code> ProcessContextImpl::setDynamicProperty(std::string name, std::string value) {
return getProcessor().setDynamicProperty(std::move(name), std::move(value));
}
+[[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> ProcessContextImpl::getAllPropertyValues(std::string_view name) const {
+ return getProcessor().getAllPropertyValues(name);
+}
+
+void ProcessContextImpl::addAutoTerminatedRelationship(const core::Relationship& relationship) {
+ return getProcessor().addAutoTerminatedRelationship(relationship);
+}
+
+bool ProcessContextImpl::isRunning() const {
+ return getProcessor().isRunning();
+}
+
StateManager* ProcessContextImpl::getStateManager() {
if (state_storage_ == nullptr) { return nullptr; }
if (!state_manager_) { state_manager_ = state_storage_->getStateManager(processor_); }
return state_manager_.get();
}
+bool ProcessContextImpl::hasIncomingConnections() const {
+ return getProcessor().hasIncomingConnections();
+}
+
} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/src/core/Processor.cpp b/libminifi/src/core/Processor.cpp
index 538c265fb..c195b3191 100644
--- a/libminifi/src/core/Processor.cpp
+++ b/libminifi/src/core/Processor.cpp
@@ -52,7 +52,8 @@ ProcessorProxy::ProcessorProxy(std::string_view name, std::unique_ptr<ProcessorA
run_duration_(DEFAULT_RUN_DURATION),
yield_period_(DEFAULT_YIELD_PERIOD_SECONDS),
active_tasks_(0),
- logger_(logging::LoggerFactory<Processor>::getLogger(uuid_)) {
+ logger_(logging::LoggerFactory<Processor>::getLogger(uuid_)),
+ impl_(std::move(impl)) {
has_work_.store(false);
// Setup the default values
strategy_ = TIMER_DRIVEN;
@@ -69,7 +70,8 @@ ProcessorProxy::ProcessorProxy(std::string_view name, const utils::Identifier& u
run_duration_(DEFAULT_RUN_DURATION),
yield_period_(DEFAULT_YIELD_PERIOD_SECONDS),
active_tasks_(0),
- logger_(logging::LoggerFactory<Processor>::getLogger(uuid_)) {
+ logger_(logging::LoggerFactory<Processor>::getLogger(uuid_)),
+ impl_(std::move(impl)) {
has_work_.store(false);
// Setup the default values
strategy_ = TIMER_DRIVEN;
@@ -183,7 +185,7 @@ void ProcessorProxy::triggerAndCommit(const std::shared_ptr<ProcessContext>& con
}
void ProcessorProxy::trigger(const std::shared_ptr<ProcessContext>& context, const std::shared_ptr<ProcessSession>& process_session) {
- ++impl_->getMetrics()->iterations();
+ ++impl_->getMetrics()->invocations();
const auto start = std::chrono::steady_clock::now();
onTrigger(*context, *process_session);
impl_->getMetrics()->addLastOnTriggerRuntime(std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::steady_clock::now() - start));
@@ -361,7 +363,7 @@ class ProcessorDescriptorImpl : public ProcessorDescriptor {
impl_->setSupportedRelationships(relationships);
}
- void setSupportedProperties(std::span<const PropertyReference> properties) {
+ void setSupportedProperties(std::span<const PropertyReference> properties) override {
impl_->setSupportedProperties(properties);
}
diff --git a/libminifi/src/core/flow/StructuredConfiguration.cpp b/libminifi/src/core/flow/StructuredConfiguration.cpp
index e056c595e..61f8d8582 100644
--- a/libminifi/src/core/flow/StructuredConfiguration.cpp
+++ b/libminifi/src/core/flow/StructuredConfiguration.cpp
@@ -24,6 +24,7 @@
#include "Funnel.h"
#include "core/ParameterContext.h"
+#include "core/ProcessorProxy.h"
#include "core/ParameterTokenParser.h"
#include "core/ReferenceParser.h"
#include "core/flow/CheckRequiredField.h"
@@ -581,7 +582,9 @@ void StructuredConfiguration::parseProvenanceReporting(const Node& node, core::P
return;
}
- auto reportTask = createProvenanceReportTask();
+ auto report_task_impl = createProvenanceReportTask();
+ auto* reportTask = report_task_impl.get();
+ auto report_task_wrapper = std::make_unique<core::ProcessorProxy>("", std::move(report_task_impl));
checkRequiredField(node, schema_.scheduling_strategy);
auto schedulingStrategyStr = node[schema_.scheduling_strategy].getString().value();
@@ -590,11 +593,11 @@ void StructuredConfiguration::parseProvenanceReporting(const Node& node, core::P
if (auto scheduling_period = utils::timeutils::StringToDuration<std::chrono::nanoseconds>(schedulingPeriodStr)) {
logger_->log_debug("ProvenanceReportingTask schedulingPeriod {}", scheduling_period);
- reportTask->setSchedulingPeriod(*scheduling_period);
+ report_task_wrapper->setSchedulingPeriod(*scheduling_period);
}
if (schedulingStrategyStr == "TIMER_DRIVEN") {
- reportTask->setSchedulingStrategy(core::TIMER_DRIVEN);
+ report_task_wrapper->setSchedulingStrategy(core::TIMER_DRIVEN);
logger_->log_debug("ProvenanceReportingTask scheduling strategy {}", schedulingStrategyStr);
} else {
throw std::invalid_argument("Invalid scheduling strategy " + schedulingStrategyStr);
@@ -631,11 +634,11 @@ void StructuredConfiguration::parseProvenanceReporting(const Node& node, core::P
reportTask->setBatchSize(*batch_size);
}
- reportTask->initialize();
+ report_task_wrapper->initialize();
// add processor to parent
- reportTask->setScheduledState(core::RUNNING);
- parent_group->addProcessor(std::move(reportTask));
+ report_task_wrapper->setScheduledState(core::RUNNING);
+ parent_group->addProcessor(std::move(report_task_wrapper));
}
void StructuredConfiguration::parseControllerServices(const Node& controller_services_node, core::ProcessGroup* parent_group) {
@@ -741,13 +744,15 @@ void StructuredConfiguration::parseRPGPort(const Node& port_node, core::ProcessG
"This is a UUID of the format XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX.");
uuid = portId;
- auto port = std::make_unique<minifi::RemoteProcessorGroupPort>(
+ auto port_impl = std::make_unique<minifi::RemoteProcessorGroupPort>(
nameStr, parent->getURL(), this->configuration_, uuid);
+ auto* port = port_impl.get();
+ auto port_wrapper = std::make_unique<core::ProcessorProxy>(nameStr, uuid, std::move(port_impl));
port->setDirection(direction);
port->setTimeout(parent->getTimeout());
port->setTransmitting(true);
- port->setYieldPeriodMsec(parent->getYieldPeriodMsec());
- port->initialize();
+ port_wrapper->setYieldPeriodMsec(parent->getYieldPeriodMsec());
+ port_wrapper->initialize();
if (!parent->getInterface().empty())
port->setInterface(parent->getInterface());
if (parent->getTransportProtocol() == "HTTP") {
@@ -759,15 +764,15 @@ void StructuredConfiguration::parseRPGPort(const Node& port_node, core::ProcessG
// handle port properties
if (const Node propertiesNode = port_node[schema_.rpg_port_properties]) {
- parsePropertiesNode(propertiesNode, *port, nameStr, nullptr);
+ parsePropertiesNode(propertiesNode, *port_wrapper, nameStr, nullptr);
} else {
- parsePropertyNodeElement(std::string(minifi::RemoteProcessorGroupPort::portUUID.name), port_node[schema_.rpg_port_target_id], *port, nullptr);
- validateComponentProperties(*port, nameStr, port_node.getPath());
+ parsePropertyNodeElement(std::string(minifi::RemoteProcessorGroupPort::portUUID.name), port_node[schema_.rpg_port_target_id], *port_wrapper, nullptr);
+ validateComponentProperties(*port_wrapper, nameStr, port_node.getPath());
}
// add processor to parent
- auto& processor = *port;
- parent->addProcessor(std::move(port));
+ auto& processor = *port_wrapper;
+ parent->addProcessor(std::move(port_wrapper));
processor.setScheduledState(core::RUNNING);
if (auto tasksNode = port_node[schema_.max_concurrent_tasks]) {
@@ -947,7 +952,7 @@ void StructuredConfiguration::parseFunnels(const Node& node, core::ProcessGroup*
throw Exception(ExceptionType::GENERAL_EXCEPTION, "Incorrect funnel UUID format.");
});
- auto funnel = std::make_unique<minifi::Funnel>(name, uuid.value());
+ auto funnel = std::make_unique<core::ProcessorProxy>(name, uuid.value(), std::make_unique<minifi::Funnel>(name, uuid.value()));
logger_->log_debug("Created funnel with UUID {} and name {}", id, name);
funnel->setScheduledState(core::RUNNING);
funnel->setSchedulingStrategy(core::EVENT_DRIVEN);
@@ -975,7 +980,7 @@ void StructuredConfiguration::parsePorts(const flow::Node& node, core::ProcessGr
throw Exception(ExceptionType::GENERAL_EXCEPTION, "Incorrect port UUID format.");
});
- auto port = std::make_unique<Port>(name, uuid.value(), port_type);
+ auto port = std::make_unique<PortProxy>(name, uuid.value(), std::make_unique<PortImpl>(name, uuid.value(), port_type));
logger_->log_debug("Created port UUID {} and name {}", id, name);
port->setScheduledState(core::RUNNING);
port->setSchedulingStrategy(core::EVENT_DRIVEN);
diff --git a/libminifi/src/core/flow/StructuredConnectionParser.cpp b/libminifi/src/core/flow/StructuredConnectionParser.cpp
index 85e7e28d5..728bc95f6 100644
--- a/libminifi/src/core/flow/StructuredConnectionParser.cpp
+++ b/libminifi/src/core/flow/StructuredConnectionParser.cpp
@@ -41,8 +41,7 @@ void StructuredConnectionParser::addFunnelRelationshipToConnection(minifi::Conne
return;
}
- auto& processor_ref = *processor;
- if (typeid(minifi::Funnel) == typeid(processor_ref)) {
+ if (dynamic_cast<minifi::Funnel*>(&processor->getImpl())) {
addNewRelationshipToConnection(minifi::Funnel::Success.name, connection);
}
}
diff --git a/libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp b/libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp
index 7aa5ea3a1..5e030fc46 100644
--- a/libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp
+++ b/libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp
@@ -189,7 +189,7 @@ void SiteToSiteProvenanceReportingTask::onTrigger(core::ProcessContext& context,
// we transfer the record, purge the record from DB
repo->Delete(records);
- returnProtocol(std::move(protocol_));
+ returnProtocol(context, std::move(protocol_));
}
} // namespace org::apache::nifi::minifi::core::reporting
diff --git a/libminifi/test/flow-tests/CustomProcessors.h b/libminifi/test/flow-tests/CustomProcessors.h
index d6e2bb1df..6e743561b 100644
--- a/libminifi/test/flow-tests/CustomProcessors.h
+++ b/libminifi/test/flow-tests/CustomProcessors.h
@@ -24,7 +24,7 @@
#include <string>
#include <utility>
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinition.h"
#include "core/PropertyDefinitionBuilder.h"
#include "minifi-cpp/core/PropertyValidator.h"
@@ -60,8 +60,7 @@ class ProcessorWithStatistics {
class TestProcessor : public core::ProcessorImpl, public ProcessorWithStatistics {
public:
- TestProcessor(std::string_view name, const utils::Identifier& uuid) : ProcessorImpl(name, uuid) {}
- explicit TestProcessor(const std::string& name) : ProcessorImpl(name) {}
+ using ProcessorImpl::ProcessorImpl;
static constexpr const char* Description = "Processor used for testing cycles";
static constexpr auto Properties = std::to_array<core::PropertyReference>({AppleProbability, BananaProbability});
@@ -110,8 +109,7 @@ class TestProcessor : public core::ProcessorImpl, public ProcessorWithStatistics
class TestFlowFileGenerator : public processors::GenerateFlowFile, public ProcessorWithStatistics {
public:
- TestFlowFileGenerator(std::string_view name, const utils::Identifier& uuid) : GenerateFlowFile(name, uuid) {}
- explicit TestFlowFileGenerator(const std::string& name) : GenerateFlowFile(name) {}
+ using GenerateFlowFile::GenerateFlowFile;
static constexpr const char* Description = "Processor generating files and notifying us";
diff --git a/libminifi/test/flow-tests/CycleTest.cpp b/libminifi/test/flow-tests/CycleTest.cpp
index c11cd1bad..d4c62a69e 100644
--- a/libminifi/test/flow-tests/CycleTest.cpp
+++ b/libminifi/test/flow-tests/CycleTest.cpp
@@ -111,16 +111,16 @@ TEST_CASE("Flow with two long cycle", "[FlowWithCycle]") {
auto controller = testController.controller_;
auto root = testController.root_;
- auto procGenerator = dynamic_cast<org::apache::nifi::minifi::processors::TestFlowFileGenerator*>(root->findProcessorByName("Generator"));
- auto procA = dynamic_cast<org::apache::nifi::minifi::processors::TestProcessor*>(root->findProcessorByName("A"));
- auto procB = dynamic_cast<org::apache::nifi::minifi::processors::TestProcessor*>(root->findProcessorByName("B"));
+ TypedProcessorWrapper<org::apache::nifi::minifi::processors::TestFlowFileGenerator> procGenerator = root->findProcessorByName("Generator");
+ TypedProcessorWrapper<org::apache::nifi::minifi::processors::TestProcessor> procA = root->findProcessorByName("A");
+ TypedProcessorWrapper<org::apache::nifi::minifi::processors::TestProcessor> procB = root->findProcessorByName("B");
int tryCount = 0;
- while (tryCount++ < 10 && (procA->trigger_count.load() < 10 || procB->trigger_count.load() < 10)) {
+ while (tryCount++ < 10 && (procA.get().trigger_count.load() < 10 || procB.get().trigger_count.load() < 10)) {
std::this_thread::sleep_for(std::chrono::milliseconds{1000});
}
- REQUIRE(procGenerator->trigger_count.load() <= 3);
- REQUIRE(procA->trigger_count.load() >= 10);
- REQUIRE(procB->trigger_count.load() >= 10);
+ REQUIRE(procGenerator.get().trigger_count.load() <= 3);
+ REQUIRE(procA.get().trigger_count.load() >= 10);
+ REQUIRE(procB.get().trigger_count.load() >= 10);
}
diff --git a/libminifi/test/flow-tests/FlowControllerTests.cpp b/libminifi/test/flow-tests/FlowControllerTests.cpp
index 5771e41f7..1acf833a5 100644
--- a/libminifi/test/flow-tests/FlowControllerTests.cpp
+++ b/libminifi/test/flow-tests/FlowControllerTests.cpp
@@ -109,7 +109,7 @@ TEST_CASE("Flow shutdown drains connections", "[TestFlow1]") {
testController.configuration_->set(minifi::Configure::nifi_flowcontroller_drain_timeout, "100 ms");
- auto sinkProc = dynamic_cast<minifi::processors::TestProcessor*>(root->findProcessorByName("TestProcessor"));
+ TypedProcessorWrapper<minifi::processors::TestProcessor> sinkProc = root->findProcessorByName("TestProcessor");
gsl_Assert(sinkProc);
// prevent execution of the consumer processor
sinkProc->yield(10s);
@@ -131,7 +131,7 @@ TEST_CASE("Flow shutdown drains connections", "[TestFlow1]") {
controller->stop();
- REQUIRE(sinkProc->trigger_count == 0);
+ REQUIRE(sinkProc.get().trigger_count == 0);
for (auto& it : connectionMap) {
REQUIRE(it.second->isEmpty());
@@ -145,12 +145,12 @@ TEST_CASE("Flow shutdown waits for a while", "[TestFlow2]") {
testController.configuration_->set(minifi::Configure::nifi_flowcontroller_drain_timeout, "10 s");
- auto sourceProc = dynamic_cast<minifi::processors::TestFlowFileGenerator*>(root->findProcessorByName("Generator"));
- auto sinkProc = dynamic_cast<minifi::processors::TestProcessor*>(root->findProcessorByName("TestProcessor"));
+ TypedProcessorWrapper<minifi::processors::TestFlowFileGenerator> sourceProc = root->findProcessorByName("Generator");
+ TypedProcessorWrapper<minifi::processors::TestProcessor> sinkProc = root->findProcessorByName("TestProcessor");
std::promise<void> execSinkPromise;
std::future<void> execSinkFuture = execSinkPromise.get_future();
- sinkProc->onTriggerCb_ = [&] {
+ sinkProc.get().onTriggerCb_ = [&] {
execSinkFuture.wait();
};
@@ -160,13 +160,13 @@ TEST_CASE("Flow shutdown waits for a while", "[TestFlow2]") {
auto flowFilesEnqueued = [&] { return root->getTotalFlowFileCount() >= 3; };
REQUIRE(verifyWithBusyWait(std::chrono::milliseconds{500}, flowFilesEnqueued));
- REQUIRE(sourceProc->trigger_count.load() >= 1);
+ REQUIRE(sourceProc.get().trigger_count.load() >= 1);
execSinkPromise.set_value();
controller->stop();
- REQUIRE(sourceProc->trigger_count.load() >= 1);
- REQUIRE(sinkProc->trigger_count.load() >= 3);
+ REQUIRE(sourceProc.get().trigger_count.load() >= 1);
+ REQUIRE(sinkProc.get().trigger_count.load() >= 3);
}
TEST_CASE("Flow stopped after grace period", "[TestFlow3]") {
@@ -176,12 +176,12 @@ TEST_CASE("Flow stopped after grace period", "[TestFlow3]") {
testController.configuration_->set(minifi::Configure::nifi_flowcontroller_drain_timeout, "1000 ms");
- auto sourceProc = dynamic_cast<minifi::processors::TestFlowFileGenerator*>(root->findProcessorByName("Generator"));
- auto sinkProc = dynamic_cast<minifi::processors::TestProcessor*>(root->findProcessorByName("TestProcessor"));
+ TypedProcessorWrapper<minifi::processors::TestFlowFileGenerator> sourceProc = root->findProcessorByName("Generator");
+ TypedProcessorWrapper<minifi::processors::TestProcessor> sinkProc = root->findProcessorByName("TestProcessor");
std::promise<void> execSinkPromise;
std::future<void> execSinkFuture = execSinkPromise.get_future();
- sinkProc->onTriggerCb_ = [&]{
+ sinkProc.get().onTriggerCb_ = [&]{
execSinkFuture.wait();
static std::atomic<bool> first_onTrigger{true};
bool isFirst = true;
@@ -197,13 +197,13 @@ TEST_CASE("Flow stopped after grace period", "[TestFlow3]") {
auto flowFilesEnqueued = [&] { return root->getTotalFlowFileCount() >= 3; };
REQUIRE(verifyWithBusyWait(std::chrono::milliseconds{500}, flowFilesEnqueued));
- REQUIRE(sourceProc->trigger_count.load() >= 1);
+ REQUIRE(sourceProc.get().trigger_count.load() >= 1);
execSinkPromise.set_value();
controller->stop();
- REQUIRE(sourceProc->trigger_count.load() >= 1);
- REQUIRE(sinkProc->trigger_count.load() >= 1);
+ REQUIRE(sourceProc.get().trigger_count.load() >= 1);
+ REQUIRE(sinkProc.get().trigger_count.load() >= 1);
}
TEST_CASE("Extend the waiting period during shutdown", "[TestFlow4]") {
@@ -215,12 +215,12 @@ TEST_CASE("Extend the waiting period during shutdown", "[TestFlow4]") {
testController.configuration_->set(minifi::Configure::nifi_flowcontroller_drain_timeout, fmt::format("{}", timeout_ms));
- auto sourceProc = dynamic_cast<minifi::processors::TestFlowFileGenerator*>(root->findProcessorByName("Generator"));
- auto sinkProc = dynamic_cast<minifi::processors::TestProcessor*>(root->findProcessorByName("TestProcessor"));
+ TypedProcessorWrapper<minifi::processors::TestFlowFileGenerator> sourceProc = root->findProcessorByName("Generator");
+ TypedProcessorWrapper<minifi::processors::TestProcessor> sinkProc = root->findProcessorByName("TestProcessor");
std::promise<void> execSinkPromise;
std::future<void> execSinkFuture = execSinkPromise.get_future();
- sinkProc->onTriggerCb_ = [&]{
+ sinkProc.get().onTriggerCb_ = [&]{
execSinkFuture.wait();
static std::atomic<bool> first_onTrigger{true};
bool isFirst = true;
@@ -236,7 +236,7 @@ TEST_CASE("Extend the waiting period during shutdown", "[TestFlow4]") {
auto flowFilesEnqueued = [&] { return root->getTotalFlowFileCount() >= 3; };
REQUIRE(verifyWithBusyWait(std::chrono::milliseconds{500}, flowFilesEnqueued));
- REQUIRE(sourceProc->trigger_count.load() >= 1);
+ REQUIRE(sourceProc.get().trigger_count.load() >= 1);
std::thread shutdownThread([&]{
execSinkPromise.set_value();
@@ -260,8 +260,8 @@ TEST_CASE("Extend the waiting period during shutdown", "[TestFlow4]") {
shutdownThread.join();
- REQUIRE(sourceProc->trigger_count.load() >= 1);
- REQUIRE(sinkProc->trigger_count.load() >= 3);
+ REQUIRE(sourceProc.get().trigger_count.load() >= 1);
+ REQUIRE(sinkProc.get().trigger_count.load() >= 3);
}
TEST_CASE("FlowController destructor releases resources", "[TestFlow5]") {
diff --git a/libminifi/test/flow-tests/LoopTest.cpp b/libminifi/test/flow-tests/LoopTest.cpp
index e46aaa2f6..e78d84fe3 100644
--- a/libminifi/test/flow-tests/LoopTest.cpp
+++ b/libminifi/test/flow-tests/LoopTest.cpp
@@ -89,17 +89,17 @@ TEST_CASE("Flow with a single loop", "[SingleLoopFlow]") {
auto controller = testController.controller_;
auto root = testController.root_;
- auto procGenerator = dynamic_cast<org::apache::nifi::minifi::processors::TestFlowFileGenerator*>(root->findProcessorByName("Generator"));
+ TypedProcessorWrapper<org::apache::nifi::minifi::processors::TestFlowFileGenerator> procGenerator = root->findProcessorByName("Generator");
gsl_Assert(procGenerator);
- auto procA = dynamic_cast<org::apache::nifi::minifi::processors::TestProcessor*>(root->findProcessorByName("A"));
+ TypedProcessorWrapper<org::apache::nifi::minifi::processors::TestProcessor> procA = root->findProcessorByName("A");
gsl_Assert(procA);
int tryCount = 0;
// wait for the procA to get triggered 15 times
- while (tryCount++ < 10 && !(procA->trigger_count.load() >= 15)) {
+ while (tryCount++ < 10 && !(procA.get().trigger_count.load() >= 15)) {
std::this_thread::sleep_for(std::chrono::milliseconds{1000});
}
- REQUIRE(procGenerator->trigger_count.load() <= 2);
- REQUIRE(procA->trigger_count.load() >= 15);
+ REQUIRE(procGenerator.get().trigger_count.load() <= 2);
+ REQUIRE(procA.get().trigger_count.load() >= 15);
}
diff --git a/libminifi/test/flow-tests/MultiLoopTest.cpp b/libminifi/test/flow-tests/MultiLoopTest.cpp
index 28aed161e..a7faa240f 100644
--- a/libminifi/test/flow-tests/MultiLoopTest.cpp
+++ b/libminifi/test/flow-tests/MultiLoopTest.cpp
@@ -98,16 +98,16 @@ TEST_CASE("Flow with two loops", "[MultiLoopFlow]") {
auto controller = testController.controller_;
auto root = testController.root_;
- auto procGenerator = dynamic_cast<org::apache::nifi::minifi::processors::TestFlowFileGenerator*>(root->findProcessorByName("Generator"));
+ TypedProcessorWrapper<org::apache::nifi::minifi::processors::TestFlowFileGenerator> procGenerator = root->findProcessorByName("Generator");
gsl_Assert(procGenerator);
- auto procA = dynamic_cast<org::apache::nifi::minifi::processors::TestProcessor*>(root->findProcessorByName("A"));
+ TypedProcessorWrapper<org::apache::nifi::minifi::processors::TestProcessor> procA = root->findProcessorByName("A");
gsl_Assert(procA);
int tryCount = 0;
- while (tryCount++ < 10 && procA->trigger_count.load() <= 15) {
+ while (tryCount++ < 10 && procA.get().trigger_count.load() <= 15) {
std::this_thread::sleep_for(std::chrono::milliseconds{1000});
}
- REQUIRE(procGenerator->trigger_count.load() <= 3);
- REQUIRE(procA->trigger_count.load() > 15);
+ REQUIRE(procGenerator.get().trigger_count.load() <= 3);
+ REQUIRE(procA.get().trigger_count.load() > 15);
}
diff --git a/libminifi/test/flow-tests/SessionTests.cpp b/libminifi/test/flow-tests/SessionTests.cpp
index 7dc99ed83..eca3b33c4 100644
--- a/libminifi/test/flow-tests/SessionTests.cpp
+++ b/libminifi/test/flow-tests/SessionTests.cpp
@@ -28,9 +28,10 @@
#include "io/BufferStream.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/RepositoryFactory.h"
#include "repository/VolatileContentRepository.h"
+#include "unit/TestUtils.h"
namespace {
@@ -79,7 +80,7 @@ TEST_CASE("Import null data") {
ff_repository->initialize(config);
content_repo->initialize(config);
- auto processor = std::make_shared<TestProcessor>("dummy");
+ auto processor = minifi::test::utils::make_processor<TestProcessor>("dummy");
utils::Identifier uuid = processor->getUUID();
auto output = std::make_unique<minifi::ConnectionImpl>(ff_repository, content_repo, "output");
output->addRelationship({"out", ""});
diff --git a/libminifi/test/integration/C2NullConfiguration.cpp b/libminifi/test/integration/C2NullConfiguration.cpp
index ec1664d66..9c1189da1 100644
--- a/libminifi/test/integration/C2NullConfiguration.cpp
+++ b/libminifi/test/integration/C2NullConfiguration.cpp
@@ -59,10 +59,8 @@ class VerifyC2Server : public HTTPIntegrationBase {
auto* const proc = pg->findProcessorByName("invoke");
REQUIRE(proc != nullptr);
- const auto* const inv = dynamic_cast<minifi::processors::InvokeHTTP*>(proc);
-
- REQUIRE(inv != nullptr);
- std::string url = inv->getProperty(processors::InvokeHTTP::URL.name).value_or("");
+ proc->getImpl<minifi::processors::InvokeHTTP>();
+ std::string url = proc->getProperty(processors::InvokeHTTP::URL.name).value_or("");
std::string port;
std::string scheme;
diff --git a/libminifi/test/integration/C2PauseResumeTest.cpp b/libminifi/test/integration/C2PauseResumeTest.cpp
index c36340ea1..03009c5b1 100644
--- a/libminifi/test/integration/C2PauseResumeTest.cpp
+++ b/libminifi/test/integration/C2PauseResumeTest.cpp
@@ -153,9 +153,8 @@ TEST_CASE("C2PauseResumeTest", "[c2test]") {
const auto proc = root->findProcessorByName("invoke");
REQUIRE(proc != nullptr);
- const auto inv = dynamic_cast<minifi::processors::InvokeHTTP*>(proc);
- REQUIRE(inv != nullptr);
- std::string url = inv->getProperty(minifi::processors::InvokeHTTP::URL.name).value_or("");
+ proc->getImpl<minifi::processors::InvokeHTTP>();
+ std::string url = proc->getProperty(minifi::processors::InvokeHTTP::URL.name).value_or("");
std::string port;
std::string scheme;
std::string path;
diff --git a/libminifi/test/integration/StateTransactionalityTests.cpp b/libminifi/test/integration/StateTransactionalityTests.cpp
index b540135ee..51721b391 100644
--- a/libminifi/test/integration/StateTransactionalityTests.cpp
+++ b/libminifi/test/integration/StateTransactionalityTests.cpp
@@ -69,9 +69,9 @@ class StatefulIntegrationTest : public IntegrationBase {
// set hooks
const auto processController = dynamic_cast<ProcessorController*>(&component);
REQUIRE(processController != nullptr);
- stateful_processor_ = dynamic_cast<StatefulProcessor*>(&processController->getProcessor());
+ stateful_processor_ = &processController->getProcessor();
REQUIRE(stateful_processor_);
- stateful_processor_->setHooks(on_schedule_hook_, on_trigger_hooks_);
+ stateful_processor_.get().setHooks(on_schedule_hook_, on_trigger_hooks_);
}
++controllerVecIdx;
@@ -83,7 +83,7 @@ class StatefulIntegrationTest : public IntegrationBase {
void runAssertions() override {
REQUIRE(utils::verifyEventHappenedInPollTime(std::chrono::milliseconds(wait_time_), [&] {
- return stateful_processor_->hasFinishedHooks() && log_checker_();
+ return stateful_processor_.get().hasFinishedHooks() && log_checker_();
}));
}
@@ -92,7 +92,7 @@ class StatefulIntegrationTest : public IntegrationBase {
const std::vector<StatefulProcessor::HookType> on_trigger_hooks_;
const LogChecker log_checker_;
const std::string test_case_;
- StatefulProcessor* stateful_processor_ = nullptr;
+ TypedProcessorWrapper<StatefulProcessor> stateful_processor_ = nullptr;
std::shared_ptr<logging::Logger> logger_{logging::LoggerFactory<StatefulIntegrationTest>::getLogger()};
};
diff --git a/libminifi/test/libtest/unit/DummyProcessor.h b/libminifi/test/libtest/unit/DummyProcessor.h
index 336c9d2e0..0e68bdfb2 100644
--- a/libminifi/test/libtest/unit/DummyProcessor.h
+++ b/libminifi/test/libtest/unit/DummyProcessor.h
@@ -27,11 +27,16 @@
namespace org::apache::nifi::minifi::test {
class DummyProcessor : public minifi::core::ProcessorImpl {
- using minifi::core::ProcessorImpl::ProcessorImpl;
-
public:
- DummyProcessor(std::string_view name, const minifi::utils::Identifier& uuid) : ProcessorImpl(name, uuid) {}
- explicit DummyProcessor(std::string_view name) : ProcessorImpl(name) {}
+ using ProcessorImpl::ProcessorImpl;
+
+ DummyProcessor(std::string_view name)
+ : ProcessorImpl{minifi::core::ProcessorMetadata{
+ .uuid = minifi::utils::IdGenerator::getIdGenerator()->generate(),
+ .name = std::string{name},
+ .logger = minifi::core::logging::LoggerFactory<DummyProcessor>::getLogger()
+ }} {}
+
static constexpr const char* Description = "A processor that does nothing.";
static constexpr auto SimpleProperty = core::PropertyDefinitionBuilder<>::createProperty("Simple Property")
.withDescription("Just a simple string property")
diff --git a/libminifi/test/libtest/unit/MockClasses.h b/libminifi/test/libtest/unit/MockClasses.h
index 73aba07ec..77c63e5ba 100644
--- a/libminifi/test/libtest/unit/MockClasses.h
+++ b/libminifi/test/libtest/unit/MockClasses.h
@@ -21,7 +21,7 @@
#include <utility>
#include "core/controller/ControllerService.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessContext.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
@@ -83,13 +83,8 @@ class MockControllerService : public minifi::core::controller::ControllerService
class MockProcessor : public minifi::core::ProcessorImpl {
public:
- explicit MockProcessor(std::string_view name, const minifi::utils::Identifier &uuid)
- : ProcessorImpl(name, uuid) {
- setTriggerWhenEmpty(true);
- }
-
- explicit MockProcessor(std::string_view name)
- : ProcessorImpl(name) {
+ explicit MockProcessor(minifi::core::ProcessorMetadata info)
+ : ProcessorImpl(info) {
setTriggerWhenEmpty(true);
}
@@ -120,7 +115,7 @@ class MockProcessor : public minifi::core::ProcessorImpl {
if (!flow_file) {
flow_file = session.create();
}
- std::string linked_service = getProperty("linkedService").value_or("");
+ std::string linked_service = context.getProperty("linkedService").value_or("");
if (!IsNullOrEmpty(linked_service)) {
std::shared_ptr<minifi::core::controller::ControllerService> service = context.getControllerService(linked_service, getUUID());
std::lock_guard<std::mutex> lock(control_mutex);
@@ -130,7 +125,7 @@ class MockProcessor : public minifi::core::ProcessorImpl {
// and verify that we can execute it.
}
- bool in_sub_process_group = getProperty("InSubProcessGroup") | minifi::utils::andThen(minifi::parsing::parseBool) | minifi::utils::orThrow("");
+ bool in_sub_process_group = context.getProperty("InSubProcessGroup") | minifi::utils::andThen(minifi::parsing::parseBool) | minifi::utils::expect("");
auto sub_service = context.getControllerService("SubMockController", getUUID());
if (in_sub_process_group) {
REQUIRE(nullptr != sub_service);
@@ -141,8 +136,4 @@ class MockProcessor : public minifi::core::ProcessorImpl {
}
session.transfer(flow_file, Success);
}
-
- bool isYield() override {
- return false;
- }
};
diff --git a/libminifi/test/libtest/unit/ReadFromFlowFileTestProcessor.h b/libminifi/test/libtest/unit/ReadFromFlowFileTestProcessor.h
index ac055b5ef..b882e028c 100644
--- a/libminifi/test/libtest/unit/ReadFromFlowFileTestProcessor.h
+++ b/libminifi/test/libtest/unit/ReadFromFlowFileTestProcessor.h
@@ -40,10 +40,7 @@ class ReadFromFlowFileTestProcessor : public core::ProcessorImpl {
static constexpr const char* ON_TRIGGER_LOG_STR = "ReadFromFlowFileTestProcessor::onTrigger executed";
static constexpr const char* ON_UNSCHEDULE_LOG_STR = "ReadFromFlowFileTestProcessor::onUnSchedule executed";
- explicit ReadFromFlowFileTestProcessor(const std::string_view name, const utils::Identifier& uuid = utils::Identifier())
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<ReadFromFlowFileTestProcessor>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
static constexpr const char* Description = "ReadFromFlowFileTestProcessor (only for testing purposes)";
diff --git a/libminifi/test/libtest/unit/SingleProcessorTestController.h b/libminifi/test/libtest/unit/SingleProcessorTestController.h
index beca2995c..a12caa9a5 100644
--- a/libminifi/test/libtest/unit/SingleProcessorTestController.h
+++ b/libminifi/test/libtest/unit/SingleProcessorTestController.h
@@ -24,6 +24,7 @@
#include <vector>
#include "TestBase.h"
#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
using namespace std::literals::chrono_literals;
@@ -61,8 +62,10 @@ class SingleProcessorTestController : public TestController {
core::Relationship addDynamicRelationship(std::string name);
- template<typename T = core::Processor>
- T* getProcessor() const { return dynamic_cast<T*>(processor_); }
+ template<typename T>
+ TypedProcessorWrapper<T> getProcessor() const { return processor_; }
+
+ core::Processor* getProcessor() const { return processor_; }
std::shared_ptr<TestPlan> plan = createPlan();
diff --git a/libminifi/test/libtest/unit/TestBase.h b/libminifi/test/libtest/unit/TestBase.h
index 86f2598b3..67a155675 100644
--- a/libminifi/test/libtest/unit/TestBase.h
+++ b/libminifi/test/libtest/unit/TestBase.h
@@ -201,6 +201,39 @@ class TempDirectory {
bool is_owner_;
};
+template<typename T>
+class TypedProcessorWrapper {
+ public:
+ TypedProcessorWrapper() = default;
+ TypedProcessorWrapper(core::Processor* proc): proc_(proc) {
+ if (proc_) {
+ proc_->getImpl<T>();
+ }
+ }
+
+ explicit operator bool() const {
+ return proc_ != nullptr;
+ }
+
+ operator core::Processor*() const {
+ gsl_Assert(proc_);
+ return proc_;
+ }
+
+ T& get() const {
+ gsl_Assert(proc_);
+ return proc_->getImpl<T>();
+ }
+
+ core::Processor* operator->() const {
+ gsl_Assert(proc_);
+ return proc_;
+ }
+
+ private:
+ core::Processor* proc_{nullptr};
+};
+
class TestPlan {
public:
explicit TestPlan(std::shared_ptr<minifi::core::ContentRepository> content_repo, std::shared_ptr<minifi::core::Repository> flow_repo, std::shared_ptr<minifi::core::Repository> prov_repo,
@@ -208,6 +241,16 @@ class TestPlan {
virtual ~TestPlan();
+ template<typename T>
+ TypedProcessorWrapper<T> addProcessor(const std::string& name) {
+ auto processor_type = std::string{core::className<T>()};
+ auto pos = processor_type.find_last_of(':');
+ if (pos != std::string::npos) {
+ processor_type = processor_type.substr(pos + 1);
+ }
+ return TypedProcessorWrapper<T>{addProcessor(processor_type, name)};
+ }
+
minifi::core::Processor* addProcessor(std::unique_ptr<minifi::core::Processor> processor, const std::string &name,
const minifi::core::Relationship& relationship = minifi::core::Relationship("success", "description"), bool linkToPrevious = false) {
return addProcessor(std::move(processor), name, { relationship }, linkToPrevious);
diff --git a/libminifi/test/libtest/unit/TestUtils.h b/libminifi/test/libtest/unit/TestUtils.h
index 7aacc538f..2c2ef7bcb 100644
--- a/libminifi/test/libtest/unit/TestUtils.h
+++ b/libminifi/test/libtest/unit/TestUtils.h
@@ -39,6 +39,8 @@
#include "asio/ssl.hpp"
#include "utils/net/Ssl.h"
#include "range/v3/algorithm/any_of.hpp"
+#include "core/ProcessorProxy.h"
+#include "core/logging/LoggerFactory.h"
using namespace std::literals::chrono_literals;
@@ -210,18 +212,17 @@ inline std::error_code hide_file(const std::filesystem::path& file_name) {
#endif /* WIN32 */
template<typename T>
-concept NetworkingProcessor = std::derived_from<T, minifi::core::Processor>
- && requires(T x) {
- {T::Port} -> std::convertible_to<core::PropertyReference>;
- {x.getPort()} -> std::convertible_to<uint16_t>;
- }; // NOLINT(readability/braces)
+concept NetworkingProcessor = requires(T x) {
+ {T::Port} -> std::convertible_to<core::PropertyReference>;
+ {x.getPort()} -> std::convertible_to<uint16_t>;
+}; // NOLINT(readability/braces)
template<NetworkingProcessor T>
-uint16_t scheduleProcessorOnRandomPort(const std::shared_ptr<TestPlan>& test_plan, T& processor) {
- REQUIRE(processor.setProperty(T::Port.name, "0"));
- test_plan->scheduleProcessor(&processor);
- REQUIRE(verifyEventHappenedInPollTime(250ms, [&processor] { return processor.getPort() != 0; }, 20ms));
- return processor.getPort();
+uint16_t scheduleProcessorOnRandomPort(const std::shared_ptr<TestPlan>& test_plan, const TypedProcessorWrapper<T>& processor) {
+ REQUIRE(processor->setProperty(T::Port.name, "0"));
+ test_plan->scheduleProcessor(processor);
+ REQUIRE(verifyEventHappenedInPollTime(250ms, [&processor] { return processor.get().getPort() != 0; }, 20ms));
+ return processor.get().getPort();
}
inline bool runningAsUnixRoot() {
@@ -231,6 +232,28 @@ inline bool runningAsUnixRoot() {
return geteuid() == 0;
#endif
}
+
+template<typename T>
+std::unique_ptr<core::Processor> make_processor(std::string_view name, std::optional<minifi::utils::Identifier> uuid = std::nullopt) {
+ if (!uuid) {
+ uuid = minifi::utils::IdGenerator::getIdGenerator()->generate();
+ }
+ auto processor_impl = std::make_unique<T>(core::ProcessorMetadata{
+ .uuid = uuid.value(),
+ .name = std::string{name},
+ .logger = minifi::core::logging::LoggerFactory<T>::getLogger(uuid.value())
+ });
+ return std::make_unique<core::ProcessorProxy>(name, uuid.value(), std::move(processor_impl));
+}
+
+template<typename T, typename ...Args>
+std::unique_ptr<core::Processor> make_custom_processor(Args&&... args) {
+ auto processor_impl = std::make_unique<T>(std::forward<Args>(args)...);
+ auto name = processor_impl->getName();
+ auto uuid = processor_impl->getUUID();
+ return std::make_unique<core::ProcessorProxy>(name, uuid, std::move(processor_impl));
+}
+
} // namespace org::apache::nifi::minifi::test::utils
namespace Catch {
diff --git a/libminifi/test/libtest/unit/WriteToFlowFileTestProcessor.h b/libminifi/test/libtest/unit/WriteToFlowFileTestProcessor.h
index 6f0b13ba9..2f875e1f9 100644
--- a/libminifi/test/libtest/unit/WriteToFlowFileTestProcessor.h
+++ b/libminifi/test/libtest/unit/WriteToFlowFileTestProcessor.h
@@ -20,7 +20,7 @@
#include <utility>
#include "core/Core.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/ProcessSession.h"
#include "core/PropertyDefinition.h"
#include "core/RelationshipDefinition.h"
@@ -38,10 +38,7 @@ class WriteToFlowFileTestProcessor : public core::ProcessorImpl {
static constexpr const char* ON_TRIGGER_LOG_STR = "WriteToFlowFileTestProcessor::onTrigger executed";
static constexpr const char* ON_UNSCHEDULE_LOG_STR = "WriteToFlowFileTestProcessor::onUnSchedule executed";
- explicit WriteToFlowFileTestProcessor(const std::string_view name, const utils::Identifier& uuid = utils::Identifier())
- : ProcessorImpl(name, uuid) {
- logger_ = core::logging::LoggerFactory<WriteToFlowFileTestProcessor>::getLogger(uuid_);
- }
+ using ProcessorImpl::ProcessorImpl;
static constexpr const char* Description = "WriteToFlowFileTestProcessor (only for testing purposes)";
static constexpr auto Properties = std::array<core::PropertyReference, 0>{};
diff --git a/libminifi/test/persistence-tests/PersistenceTests.cpp b/libminifi/test/persistence-tests/PersistenceTests.cpp
index 40d9ef3bf..4518e916c 100644
--- a/libminifi/test/persistence-tests/PersistenceTests.cpp
+++ b/libminifi/test/persistence-tests/PersistenceTests.cpp
@@ -36,6 +36,7 @@
#include "../../extensions/rocksdb-repos/DatabaseContentRepository.h"
#include "unit/TestUtils.h"
#include "core/repository/FileSystemRepository.h"
+#include "core/ProcessorImpl.h"
using ConnectionImpl = minifi::ConnectionImpl;
using Connection = minifi::Connection;
@@ -69,7 +70,7 @@ struct TestFlow{
// setup INPUT processor
{
- inputProcessor = std::make_shared<TestProcessor>("source", inputProcUUID());
+ inputProcessor = minifi::test::utils::make_processor<TestProcessor>("source", inputProcUUID());
inputContext = std::make_shared<core::ProcessContextImpl>(*inputProcessor, nullptr, prov_repo,
ff_repository, content_repo);
}
@@ -144,8 +145,8 @@ struct TestFlow{
std::unique_ptr<core::ProcessSessionFactory> process_session_factory_;
};
-std::unique_ptr<MergeContent> setupMergeProcessor(const utils::Identifier& id) {
- auto processor = std::make_unique<MergeContent>("MergeContent", id);
+std::unique_ptr<core::Processor> setupMergeProcessor(const utils::Identifier& id) {
+ auto processor = minifi::test::utils::make_processor<MergeContent>("MergeContent", id);
processor->initialize();
processor->setAutoTerminatedRelationships(std::array{core::Relationship{"original", "d"}});
@@ -252,7 +253,7 @@ TEST_CASE("Processors Can Store FlowFiles", "[TestP1]") {
class ContentUpdaterProcessor : public core::ProcessorImpl {
public:
- ContentUpdaterProcessor(std::string_view name, const utils::Identifier& id) : ProcessorImpl(name, id) {}
+ using ProcessorImpl::ProcessorImpl;
static constexpr bool SupportsDynamicProperties = false;
static constexpr bool SupportsDynamicRelationships = false;
@@ -270,7 +271,7 @@ class ContentUpdaterProcessor : public core::ProcessorImpl {
};
std::unique_ptr<core::Processor> setupContentUpdaterProcessor(const utils::Identifier& id) {
- return std::make_unique<ContentUpdaterProcessor>("Updater", id);
+ return minifi::test::utils::make_processor<ContentUpdaterProcessor>("Updater", id);
}
TEST_CASE("Persisted flowFiles are updated on modification", "[TestP1]") {
diff --git a/libminifi/test/unit/AbstractProcessorTest.cpp b/libminifi/test/unit/AbstractProcessorTest.cpp
index df2de201a..0571ef31d 100644
--- a/libminifi/test/unit/AbstractProcessorTest.cpp
+++ b/libminifi/test/unit/AbstractProcessorTest.cpp
@@ -20,10 +20,12 @@
#include "unit/Catch.h"
#include "core/AbstractProcessor.h"
#include "core/PropertyDefinitionBuilder.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::test {
struct AbstractProcessorTestCase1 : core::AbstractProcessor<AbstractProcessorTestCase1> {
+ using AbstractProcessor::AbstractProcessor;
static constexpr auto SupportsDynamicProperties = true;
static constexpr auto SupportsDynamicRelationships = true;
static constexpr core::annotation::Input InputRequirement = core::annotation::Input::INPUT_FORBIDDEN;
@@ -41,13 +43,13 @@ struct AbstractProcessorTestCase1 : core::AbstractProcessor<AbstractProcessorTes
static constexpr auto Properties = std::to_array<core::PropertyReference>({Property1, Property2});
static constexpr core::RelationshipDefinition Rel1{"rel1", "rel1 description"};
static constexpr auto Relationships = std::array{Rel1};
- AbstractProcessorTestCase1() :core::AbstractProcessor<AbstractProcessorTestCase1>{"TestCase1"} {}
void onSchedule(core::ProcessContext&, core::ProcessSessionFactory&) override {}
void onTrigger(core::ProcessContext&, core::ProcessSession&) override {}
};
TEST_CASE("AbstractProcessor case1", "[processor][abstractprocessor][case1]") {
- AbstractProcessorTestCase1 processor;
+ auto proc = minifi::test::utils::make_processor<AbstractProcessorTestCase1>("TestCase1");
+ auto& processor = *proc;
processor.initialize();
REQUIRE(processor.getName() == "TestCase1");
REQUIRE(processor.supportsDynamicProperties() == true);
@@ -68,6 +70,7 @@ TEST_CASE("AbstractProcessor case1", "[processor][abstractprocessor][case1]") {
}
struct AbstractProcessorTestCase2 : core::AbstractProcessor<AbstractProcessorTestCase2> {
+ using AbstractProcessor::AbstractProcessor;
static constexpr bool SupportsDynamicProperties = false;
static constexpr bool SupportsDynamicRelationships = false;
static constexpr core::annotation::Input InputRequirement = core::annotation::Input::INPUT_REQUIRED;
@@ -85,13 +88,13 @@ struct AbstractProcessorTestCase2 : core::AbstractProcessor<AbstractProcessorTes
static constexpr auto Properties = std::to_array<core::PropertyReference>({Property1, Property2});
static constexpr core::RelationshipDefinition Rel1{"Relationship1", "rel1 description"};
static constexpr auto Relationships = std::array{Rel1};
- AbstractProcessorTestCase2() :core::AbstractProcessor<AbstractProcessorTestCase2>{"TestCase2"} {}
void onSchedule(core::ProcessContext&, core::ProcessSessionFactory&) override {}
void onTrigger(core::ProcessContext&, core::ProcessSession&) override {}
};
TEST_CASE("AbstractProcessor case2", "[processor][abstractprocessor][case2]") {
- AbstractProcessorTestCase2 processor;
+ auto proc = minifi::test::utils::make_processor<AbstractProcessorTestCase2>("TestCase2");
+ auto& processor = *proc;
processor.initialize();
REQUIRE(processor.getName() == "TestCase2");
REQUIRE(!processor.supportsDynamicProperties());
diff --git a/libminifi/test/unit/ComponentManifestTests.cpp b/libminifi/test/unit/ComponentManifestTests.cpp
index 5cfd2e27f..5fba412bc 100644
--- a/libminifi/test/unit/ComponentManifestTests.cpp
+++ b/libminifi/test/unit/ComponentManifestTests.cpp
@@ -26,6 +26,7 @@
#include "core/PropertyDefinitionBuilder.h"
#include "core/Resource.h"
#include "core/state/nodes/AgentInformation.h"
+#include "core/ProcessorImpl.h"
using minifi::state::response::SerializedResponseNode;
diff --git a/libminifi/test/unit/MetricsTests.cpp b/libminifi/test/unit/MetricsTests.cpp
index 806970b07..c72c3e951 100644
--- a/libminifi/test/unit/MetricsTests.cpp
+++ b/libminifi/test/unit/MetricsTests.cpp
@@ -29,6 +29,7 @@
#include "range/v3/algorithm/find_if.hpp"
#include "unit/SingleProcessorTestController.h"
#include "core/ProcessorMetrics.h"
+#include "unit/TestUtils.h"
using namespace std::literals::chrono_literals;
@@ -288,11 +289,8 @@ TEST_CASE("Test commit runtime processor metrics", "[ProcessorMetrics]") {
}
class DuplicateContentProcessor : public minifi::core::ProcessorImpl {
- using minifi::core::ProcessorImpl::ProcessorImpl;
-
public:
- DuplicateContentProcessor(std::string_view name, const minifi::utils::Identifier& uuid) : ProcessorImpl(name, uuid) {}
- explicit DuplicateContentProcessor(std::string_view name) : ProcessorImpl(name) {}
+ using ProcessorImpl::ProcessorImpl;
static constexpr const char* Description = "A processor that creates two more of the same flow file.";
static constexpr auto Properties = std::array<core::PropertyReference, 0>{};
static constexpr auto Success = core::RelationshipDefinition{"success", "Newly created FlowFiles"};
@@ -330,7 +328,7 @@ class DuplicateContentProcessor : public minifi::core::ProcessorImpl {
};
TEST_CASE("Test processor metrics change after trigger", "[ProcessorMetrics]") {
- minifi::test::SingleProcessorTestController test_controller(std::make_unique<DuplicateContentProcessor>("DuplicateContentProcessor"));
+ minifi::test::SingleProcessorTestController test_controller(minifi::test::utils::make_processor<DuplicateContentProcessor>("DuplicateContentProcessor"));
test_controller.trigger({minifi::test::InputFlowFileData{"log line 1", {}}});
auto metrics = test_controller.getProcessor()->getMetrics();
CHECK(metrics->invocations() == 1);
diff --git a/libminifi/test/unit/ProcessorConfigUtilsTests.cpp b/libminifi/test/unit/ProcessorConfigUtilsTests.cpp
index 7336fad77..9e03584f0 100644
--- a/libminifi/test/unit/ProcessorConfigUtilsTests.cpp
+++ b/libminifi/test/unit/ProcessorConfigUtilsTests.cpp
@@ -18,10 +18,11 @@
#include "unit/TestBase.h"
#include "unit/Catch.h"
#include "core/PropertyDefinition.h"
-#include "core/Processor.h"
+#include "core/ProcessorImpl.h"
#include "core/PropertyDefinitionBuilder.h"
#include "utils/ProcessorConfigUtils.h"
#include "utils/Enum.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::core {
namespace {
@@ -30,11 +31,17 @@ class TestProcessor : public ProcessorImpl {
public:
using ProcessorImpl::ProcessorImpl;
+ void initialize() override {
+ on_initialize_(*this);
+ }
+
static constexpr bool SupportsDynamicProperties = false;
static constexpr bool SupportsDynamicRelationships = false;
static constexpr core::annotation::Input InputRequirement = core::annotation::Input::INPUT_ALLOWED;
static constexpr bool IsSingleThreaded = false;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
+
+ std::function<void(ProcessorImpl&)> on_initialize_;
};
enum class TestEnum {
@@ -46,8 +53,11 @@ TEST_CASE("Parse enum property") {
static constexpr auto prop = PropertyDefinitionBuilder<magic_enum::enum_count<TestEnum>()>::createProperty("prop")
.withAllowedValues(magic_enum::enum_names<TestEnum>())
.build();
- auto proc = std::make_shared<TestProcessor>("test-proc");
- proc->setSupportedProperties(std::to_array<core::PropertyReference>({prop}));
+ auto proc = minifi::test::utils::make_processor<TestProcessor>("test-proc");
+ proc->getImpl<TestProcessor>().on_initialize_ = [&] (auto& self) {
+ self.setSupportedProperties(std::to_array<core::PropertyReference>({prop}));
+ };
+ proc->initialize();
ProcessContextImpl context(*proc, nullptr, nullptr, nullptr, nullptr, nullptr);
SECTION("Valid") {
REQUIRE(proc->setProperty(prop.name, "B"));
diff --git a/libminifi/test/unit/ResponseNodeLoaderTests.cpp b/libminifi/test/unit/ResponseNodeLoaderTests.cpp
index 23accca8e..1ceb164b4 100644
--- a/libminifi/test/unit/ResponseNodeLoaderTests.cpp
+++ b/libminifi/test/unit/ResponseNodeLoaderTests.cpp
@@ -25,6 +25,7 @@
#include "core/repository/VolatileContentRepository.h"
#include "utils/Id.h"
#include "unit/ProvenanceTestHelper.h"
+#include "unit/TestUtils.h"
namespace org::apache::nifi::minifi::test {
@@ -48,9 +49,9 @@ class ResponseNodeLoaderTestFixture {
}
protected:
- template<typename T, typename = typename std::enable_if_t<std::is_base_of_v<minifi::core::Processor, T>>>
+ template<typename T, typename = typename std::enable_if_t<std::is_base_of_v<minifi::core::ProcessorImpl, T>>>
minifi::utils::Identifier addProcessor(const std::string& name) {
- auto processor = std::make_unique<T>(name);
+ auto processor = minifi::test::utils::make_processor<T>(name);
auto uuid = processor->getUUID();
root_->addProcessor(std::move(processor));
return uuid;
diff --git a/libminifi/test/unit/SchedulingAgentTests.cpp b/libminifi/test/unit/SchedulingAgentTests.cpp
index 2a6a7abc8..76424a250 100644
--- a/libminifi/test/unit/SchedulingAgentTests.cpp
+++ b/libminifi/test/unit/SchedulingAgentTests.cpp
@@ -84,7 +84,8 @@ class SchedulingAgentTestFixture {
std::make_unique<minifi::core::controller::ControllerServiceNodeMap>(), configuration_);
utils::ThreadPool thread_pool_;
- std::shared_ptr<CountOnTriggersProcessor> count_proc_ = std::make_shared<CountOnTriggersProcessor>("count_proc");
+ std::shared_ptr<core::Processor> count_proc_ = minifi::test::utils::make_processor<CountOnTriggersProcessor>("count_proc");
+ CountOnTriggersProcessor* count_proc_impl_ = &count_proc_->getImpl<CountOnTriggersProcessor>();
std::shared_ptr<core::ProcessContext> context_ = std::make_shared<core::ProcessContextImpl>(*count_proc_, nullptr, test_repo_, test_repo_, content_repo_);
std::shared_ptr<core::ProcessSessionFactory> factory_ = std::make_shared<core::ProcessSessionFactoryImpl>(context_);
};
@@ -100,20 +101,20 @@ TEST_CASE_METHOD(SchedulingAgentTestFixture, "TimerDrivenSchedulingAgent") {
auto first_task_reschedule_info = timer_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!first_task_reschedule_info.isFinished());
CHECK(first_task_reschedule_info.getNextExecutionTime() <= std::chrono::steady_clock::now() + 125ms);
- CHECK(count_proc_->getNumberOfTriggers() == 1);
+ CHECK(count_proc_impl_->getNumberOfTriggers() == 1);
- count_proc_->setOnTriggerDuration(50ms);
+ count_proc_impl_->setOnTriggerDuration(50ms);
auto second_task_reschedule_info = timer_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!second_task_reschedule_info.isFinished());
CHECK(first_task_reschedule_info.getNextExecutionTime() <= std::chrono::steady_clock::now() + 75ms);
- CHECK(count_proc_->getNumberOfTriggers() == 2);
+ CHECK(count_proc_impl_->getNumberOfTriggers() == 2);
- count_proc_->setOnTriggerDuration(150ms);
+ count_proc_impl_->setOnTriggerDuration(150ms);
auto third_task_reschedule_info = timer_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!third_task_reschedule_info.isFinished());
CHECK(first_task_reschedule_info.getNextExecutionTime() < std::chrono::steady_clock::now());
- CHECK(count_proc_->getNumberOfTriggers() == 3);
+ CHECK(count_proc_impl_->getNumberOfTriggers() == 3);
}
TEST_CASE_METHOD(SchedulingAgentTestFixture, "EventDrivenSchedulingAgent") {
@@ -122,13 +123,13 @@ TEST_CASE_METHOD(SchedulingAgentTestFixture, "EventDrivenSchedulingAgent") {
auto first_task_reschedule_info = event_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!first_task_reschedule_info.isFinished());
CHECK(first_task_reschedule_info.getNextExecutionTime() < std::chrono::steady_clock::now());
- auto count_num_after_one_schedule = count_proc_->getNumberOfTriggers();
+ auto count_num_after_one_schedule = count_proc_impl_->getNumberOfTriggers();
CHECK(count_num_after_one_schedule > 100);
auto second_task_reschedule_info = event_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!second_task_reschedule_info.isFinished());
CHECK(second_task_reschedule_info.getNextExecutionTime() < std::chrono::steady_clock::now());
- auto count_num_after_two_schedule = count_proc_->getNumberOfTriggers();
+ auto count_num_after_two_schedule = count_proc_impl_->getNumberOfTriggers();
CHECK(count_num_after_two_schedule > count_num_after_one_schedule+100);
}
@@ -148,13 +149,13 @@ TEST_CASE_METHOD(SchedulingAgentTestFixture, "Cron Driven every year") {
auto time_until_new_years_day = new_years_day.get_local_time() - current_time.get_local_time();
CHECK(std::chrono::abs(time_until_new_years_day - wait_time_till_next_execution_time) < 1min);
- CHECK(count_proc_->getNumberOfTriggers() == 0);
+ CHECK(count_proc_impl_->getNumberOfTriggers() == 0);
auto second_task_reschedule_info = cron_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!second_task_reschedule_info.isFinished());
CHECK(std::chrono::abs(first_task_reschedule_info.getNextExecutionTime() - second_task_reschedule_info.getNextExecutionTime()) < 1min);
- CHECK(count_proc_->getNumberOfTriggers() == 0);
+ CHECK(count_proc_impl_->getNumberOfTriggers() == 0);
}
}
@@ -165,13 +166,13 @@ TEST_CASE_METHOD(SchedulingAgentTestFixture, "Cron Driven every sec") {
auto first_task_reschedule_info = cron_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!first_task_reschedule_info.isFinished());
CHECK(first_task_reschedule_info.getNextExecutionTime() <= std::chrono::steady_clock::now() + 1s);
- CHECK(count_proc_->getNumberOfTriggers() == 0);
+ CHECK(count_proc_impl_->getNumberOfTriggers() == 0);
std::this_thread::sleep_until(first_task_reschedule_info.getNextExecutionTime());
auto second_task_reschedule_info = cron_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!second_task_reschedule_info.isFinished());
CHECK(second_task_reschedule_info.getNextExecutionTime() <= std::chrono::steady_clock::now() + 1s);
- CHECK(count_proc_->getNumberOfTriggers() == 1);
+ CHECK(count_proc_impl_->getNumberOfTriggers() == 1);
}
TEST_CASE_METHOD(SchedulingAgentTestFixture, "Cron Driven no future triggers") {
@@ -191,13 +192,13 @@ TEST_CASE_METHOD(SchedulingAgentTestFixture, "Timer driven should respect both y
count_proc_->setSchedulingPeriod(10ms);
count_proc_->setYieldPeriodMsec(1min);
}
- count_proc_->setShouldYield(true);
+ count_proc_impl_->setShouldYield(true);
auto timer_driven_agent = std::make_shared<TimerDrivenSchedulingAgent>(gsl::make_not_null(controller_services_provider_.get()), test_repo_, test_repo_, content_repo_, configuration_, thread_pool_);
timer_driven_agent->start();
auto first_task_reschedule_info = timer_driven_agent->run(count_proc_.get(), context_, factory_);
CHECK(!first_task_reschedule_info.isFinished());
CHECK(first_task_reschedule_info.getNextExecutionTime() > std::chrono::steady_clock::now() + 100ms);
- CHECK(count_proc_->getNumberOfTriggers() == 1);
+ CHECK(count_proc_impl_->getNumberOfTriggers() == 1);
}
} // namespace org::apache::nifi::minifi::testing
diff --git a/minifi-api/include/minifi-cpp/core/ConfigurableComponent.h b/minifi-api/include/minifi-cpp/core/ConfigurableComponent.h
index 244e33f3a..9f53eba70 100644
--- a/minifi-api/include/minifi-cpp/core/ConfigurableComponent.h
+++ b/minifi-api/include/minifi-cpp/core/ConfigurableComponent.h
@@ -47,6 +47,9 @@ class ConfigurableComponent : public virtual CoreComponent {
[[nodiscard]] virtual std::vector<std::string> getDynamicPropertyKeys() const = 0;
[[nodiscard]] virtual std::map<std::string, std::string> getDynamicProperties() const = 0;
+ [[nodiscard]] virtual nonstd::expected<std::vector<std::string>, std::error_code> getAllPropertyValues(std::string_view name) const = 0;
+ [[nodiscard]] virtual nonstd::expected<std::vector<std::string>, std::error_code> getAllDynamicPropertyValues(std::string_view name) const = 0;
+
[[nodiscard]] virtual bool supportsDynamicProperties() const = 0;
[[nodiscard]] virtual bool supportsDynamicRelationships() const = 0;
diff --git a/minifi-api/include/minifi-cpp/core/ProcessContext.h b/minifi-api/include/minifi-cpp/core/ProcessContext.h
index 0d319dcf9..d925d9f55 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessContext.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContext.h
@@ -56,12 +56,16 @@ class ProcessContext : public virtual core::VariableRegistry, public virtual uti
virtual const ProcessorInfo& getProcessorInfo() const = 0;
virtual Processor& getProcessor() const = 0;
+ virtual bool hasNonEmptyProperty(std::string_view name) const = 0;
+
virtual nonstd::expected<std::string, std::error_code> getProperty(std::string_view name, const FlowFile* flow_file = nullptr) const = 0;
nonstd::expected<std::string, std::error_code> getProperty(const Property& property, const FlowFile* flow_file = nullptr) const { return getProperty(property.getName(), flow_file); }
nonstd::expected<std::string, std::error_code> getProperty(const PropertyReference& property_reference, const FlowFile* flow_file = nullptr) const {
return getProperty(property_reference.name, flow_file);
}
+ virtual nonstd::expected<std::string, std::error_code> getRawProperty(std::string_view name) const = 0;
+
virtual nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value) = 0;
nonstd::expected<void, std::error_code> setProperty(const Property& property, std::string value) { return setProperty(property.getName(), std::move(value)); };
nonstd::expected<void, std::error_code> setProperty(const PropertyReference& property_reference, std::string value) { return setProperty(property_reference.name, std::move(value)); };
@@ -74,6 +78,11 @@ class ProcessContext : public virtual core::VariableRegistry, public virtual uti
virtual std::vector<std::string> getDynamicPropertyKeys() const = 0;
virtual std::map<std::string, std::string> getDynamicProperties(const FlowFile* flow_file = nullptr) const = 0;
+ [[nodiscard]] virtual nonstd::expected<std::vector<std::string>, std::error_code> getAllPropertyValues(std::string_view name) const = 0;
+
+ virtual void addAutoTerminatedRelationship(const core::Relationship& relationship) = 0;
+
+ virtual bool isRunning() const = 0;
virtual bool isAutoTerminated(Relationship relationship) const = 0;
virtual uint8_t getMaxConcurrentTasks() const = 0;
virtual void yield() = 0;
@@ -83,6 +92,8 @@ class ProcessContext : public virtual core::VariableRegistry, public virtual uti
virtual void initializeContentRepository(const std::string& home) = 0;
virtual bool isInitialized() const = 0;
+ virtual bool hasIncomingConnections() const = 0;
+
virtual std::shared_ptr<core::controller::ControllerService> getControllerService(const std::string &identifier, const utils::Identifier &processor_uuid) const = 0;
static constexpr char const* DefaultStateStorageName = "defaultstatestorage";
diff --git a/libminifi/include/Funnel.h b/minifi-api/include/minifi-cpp/core/ProcessContextApi.h
similarity index 54%
copy from libminifi/include/Funnel.h
copy to minifi-api/include/minifi-cpp/core/ProcessContextApi.h
index b6b3a7363..8e75ccdb5 100644
--- a/libminifi/include/Funnel.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessContextApi.h
@@ -1,5 +1,4 @@
/**
- *
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
@@ -18,21 +17,21 @@
#pragma once
#include <string>
-#include <utility>
-
-#include "ForwardingNode.h"
+#include <map>
+#include "utils/expected.h"
+#include "minifi-cpp/core/FlowFile.h"
+#include "minifi-cpp/core/ProcessContext.h"
-namespace org::apache::nifi::minifi {
+namespace org::apache::nifi::minifi::core {
-class Funnel final : public ForwardingNode {
+class ProcessContextApi {
public:
- Funnel(std::string_view name, const utils::Identifier& uuid) : ForwardingNode(name, uuid, core::logging::LoggerFactory<Funnel>::getLogger(uuid)) {}
- explicit Funnel(std::string_view name) : ForwardingNode(name, core::logging::LoggerFactory<Funnel>::getLogger()) {}
+ virtual nonstd::expected<std::string, std::error_code> getProperty(ProcessContext& context, std::string_view name, const FlowFile*) const = 0;
+ virtual nonstd::expected<std::string, std::error_code> getDynamicProperty(ProcessContext& context, std::string_view name, const FlowFile*) const = 0;
- MINIFIAPI static constexpr core::annotation::Input InputRequirement = core::annotation::Input::INPUT_REQUIRED;
- ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
+ virtual std::map<std::string, std::string> getDynamicProperties(ProcessContext& context, const FlowFile*) const = 0;
- void onSchedule(core::ProcessContext&, core::ProcessSessionFactory&) override;
+ virtual ~ProcessContextApi() = default;
};
-} // namespace org::apache::nifi::minifi
+} // namespace org::apache::nifi::minifi::core
\ No newline at end of file
diff --git a/utils/include/core/ConfigurableComponentImpl.h b/utils/include/core/ConfigurableComponentImpl.h
index b9049b523..cf0460c5f 100644
--- a/utils/include/core/ConfigurableComponentImpl.h
+++ b/utils/include/core/ConfigurableComponentImpl.h
@@ -49,8 +49,8 @@ class ConfigurableComponentImpl : public virtual ConfigurableComponent {
// for property sequences
nonstd::expected<void, std::error_code> appendProperty(std::string_view name, std::string value) override;
nonstd::expected<void, std::error_code> appendDynamicProperty(std::string_view name, std::string value) override;
- [[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> getAllPropertyValues(std::string_view name) const;
- [[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> getAllDynamicPropertyValues(std::string_view name) const;
+ [[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> getAllPropertyValues(std::string_view name) const override;
+ [[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> getAllDynamicPropertyValues(std::string_view name) const override;
private:
std::shared_ptr<logging::Logger> logger_ = logging::LoggerFactory<ConfigurableComponentImpl>::getLogger();
diff --git a/utils/include/core/ProcessContext.h b/utils/include/core/ProcessContext.h
index 4728236d6..df73d4cc5 100644
--- a/utils/include/core/ProcessContext.h
+++ b/utils/include/core/ProcessContext.h
@@ -62,11 +62,19 @@ class ProcessContextImpl : public core::VariableRegistryImpl, public virtual Pro
const ProcessorInfo& getProcessorInfo() const override { return *info_; }
+ virtual bool hasNonEmptyProperty(std::string_view name) const override;
+
+ bool isRunning() const override;
nonstd::expected<std::string, std::error_code> getProperty(std::string_view name, const FlowFile*) const override;
nonstd::expected<void, std::error_code> setProperty(std::string_view name, std::string value) override;
nonstd::expected<void, std::error_code> clearProperty(std::string_view name) override;
nonstd::expected<std::string, std::error_code> getDynamicProperty(std::string_view name, const FlowFile*) const override;
nonstd::expected<void, std::error_code> setDynamicProperty(std::string name, std::string value) override;
+ nonstd::expected<std::string, std::error_code> getRawProperty(std::string_view name) const override;
+ [[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> getAllPropertyValues(std::string_view name) const override;
+ bool hasIncomingConnections() const override;
+
+ void addAutoTerminatedRelationship(const core::Relationship& relationship) override;
std::vector<std::string> getDynamicPropertyKeys() const override;
std::map<std::string, std::string> getDynamicProperties(const FlowFile*) const override;
diff --git a/utils/include/core/ProcessorImpl.h b/utils/include/core/ProcessorImpl.h
index 633a80c70..225e2b243 100644
--- a/utils/include/core/ProcessorImpl.h
+++ b/utils/include/core/ProcessorImpl.h
@@ -43,7 +43,7 @@
#include "utils/Id.h"
#include "minifi-cpp/core/OutputAttributeDefinition.h"
#include "minifi-cpp/core/Processor.h"
-#include "minifi-cpp/utils/PropertyErrors.h"
+#include "utils/PropertyErrors.h"
#include "minifi-cpp/core/ProcessorMetadata.h"
#include "Exception.h"
@@ -71,14 +71,11 @@ class ProcessContext;
class ProcessSession;
class ProcessSessionFactory;
-constexpr std::chrono::microseconds MINIMUM_SCHEDULING_PERIOD{30};
-
#define BUILDING_DLL 1
class ProcessorImpl : public virtual ProcessorApi {
public:
- explicit ProcessorImpl(ProcessorMetadata info, std::shared_ptr<ProcessorMetrics> metrics = nullptr);
- explicit ProcessorImpl(std::string_view name, const utils::Identifier& uuid = utils::IdGenerator::getIdGenerator()->generate(), std::shared_ptr<core::logging::Logger> logger = nullptr, std::shared_ptr<ProcessorMetrics> metrics = nullptr);
+ explicit ProcessorImpl(ProcessorMetadata info);
ProcessorImpl(const ProcessorImpl& parent) = delete;
ProcessorImpl& operator=(const ProcessorImpl& parent) = delete;
diff --git a/utils/src/core/Connectable.cpp b/utils/src/core/Connectable.cpp
index 8a77b9547..423713236 100644
--- a/utils/src/core/Connectable.cpp
+++ b/utils/src/core/Connectable.cpp
@@ -138,6 +138,8 @@ std::set<Connectable*> ConnectableImpl::getOutGoingConnections(const std::string
const auto it = outgoing_connections_.find(relationship);
if (it != outgoing_connections_.end()) {
return it->second;
+ } else if (relationship == "__self__") {
+ return {this};
} else {
return {};
}
diff --git a/utils/src/core/Processor.cpp b/utils/src/core/Processor.cpp
index e70957e21..0caaf9c67 100644
--- a/utils/src/core/Processor.cpp
+++ b/utils/src/core/Processor.cpp
@@ -38,29 +38,20 @@
#include "fmt/format.h"
#include "Exception.h"
#include "core/ProcessorMetrics.h"
-#include "minifi-cpp/utils/PropertyErrors.h"
#include "minifi-cpp/core/ProcessorDescriptor.h"
using namespace std::literals::chrono_literals;
namespace org::apache::nifi::minifi::core {
-ProcessorImpl::ProcessorImpl(ProcessorMetadata info, std::shared_ptr<ProcessorMetrics> metrics)
+ProcessorImpl::ProcessorImpl(ProcessorMetadata info)
: info_(info),
trigger_when_empty_(false),
- metrics_(metrics ? std::move(metrics) : std::make_shared<ProcessorMetricsImpl>(*this)),
+ metrics_(std::make_shared<ProcessorMetricsImpl>(*this)),
logger_(info.logger) {
logger_->log_debug("Processor {} created with uuid {}", getName(), getUUIDStr());
}
-ProcessorImpl::ProcessorImpl(std::string_view name, const utils::Identifier& uuid, std::shared_ptr<core::logging::Logger> logger, std::shared_ptr<ProcessorMetrics> metrics)
- : ProcessorImpl(ProcessorMetadata{
- .uuid = uuid,
- .name = std::string{name},
- .logger = logger ? logger : logging::LoggerFactory<ProcessorImpl>::getLogger(uuid)
- }, metrics)
-{}
-
ProcessorImpl::~ProcessorImpl() {
logger_->log_debug("Destroying processor {} with uuid {}", getName(), getUUIDStr());
}
@@ -69,7 +60,7 @@ bool ProcessorImpl::isWorkAvailable() {
return false;
}
-void ProcessorImpl::restore(const std::shared_ptr<FlowFile>& file) {
+void ProcessorImpl::restore(const std::shared_ptr<FlowFile>& /*file*/) {
gsl_Assert("Not implemented");
}
(nifi-minifi-cpp) 14/37: Linter fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit 02f016bb6adcd46b1bc6129c7ec15c6de51e1b2f
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon May 12 11:28:31 2025 +0200
Linter fix
---
.../AzureDataLakeStorageFileProcessorBase.h | 2 +-
.../azure/tests/AzureBlobStorageTestsFixture.h | 1 -
.../azure/tests/AzureDataLakeStorageTestsFixture.h | 4 +-
.../azure/tests/ListAzureBlobStorageTests.cpp | 5 +-
.../azure/tests/ListAzureDataLakeStorageTests.cpp | 4 +-
extensions/pdh/PerformanceDataMonitor.cpp | 6 +-
libminifi/include/Funnel.h | 3 +-
libminifi/include/Port.h | 4 +-
.../reporting/SiteToSiteProvenanceReportingTask.h | 3 +-
libminifi/src/core/ClassLoader.cpp | 2 +-
libminifi/src/core/ConfigurableComponentImpl.cpp | 149 ------------------
libminifi/src/core/Connectable.cpp | 173 ---------------------
libminifi/src/core/Processor.cpp | 1 -
libminifi/test/libtest/unit/DummyProcessor.h | 2 +-
libminifi/test/libtest/unit/ProcessorUtils.h | 2 +-
libminifi/test/libtest/unit/TestBase.h | 2 +-
16 files changed, 25 insertions(+), 338 deletions(-)
diff --git a/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h b/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
index acb80568a..55d522ff5 100644
--- a/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
+++ b/extensions/azure/processors/AzureDataLakeStorageFileProcessorBase.h
@@ -41,7 +41,7 @@ class AzureDataLakeStorageFileProcessorBase : public AzureDataLakeStorageProcess
~AzureDataLakeStorageFileProcessorBase() override = default;
protected:
- explicit AzureDataLakeStorageFileProcessorBase(core::ProcessorMetadata info,std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
+ explicit AzureDataLakeStorageFileProcessorBase(core::ProcessorMetadata info, std::unique_ptr<storage::DataLakeStorageClient> data_lake_storage_client)
: AzureDataLakeStorageProcessorBase(info, std::move(data_lake_storage_client)) {
}
diff --git a/extensions/azure/tests/AzureBlobStorageTestsFixture.h b/extensions/azure/tests/AzureBlobStorageTestsFixture.h
index 897a7905f..181504303 100644
--- a/extensions/azure/tests/AzureBlobStorageTestsFixture.h
+++ b/extensions/azure/tests/AzureBlobStorageTestsFixture.h
@@ -33,7 +33,6 @@
#include "processors/UpdateAttribute.h"
#include "utils/file/FileUtils.h"
#include "MockBlobStorage.h"
-#include "core/Processor.h"
const std::string CONTAINER_NAME = "test-container";
const std::string STORAGE_ACCOUNT_NAME = "test-account";
diff --git a/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h b/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
index e7f505f3c..006ffa28c 100644
--- a/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
+++ b/extensions/azure/tests/AzureDataLakeStorageTestsFixture.h
@@ -62,7 +62,9 @@ class AzureDataLakeStorageTestsFixture {
mock_data_lake_storage_client_ptr_ = mock_data_lake_storage_client.get();
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
auto impl = std::unique_ptr<AzureDataLakeStorageProcessor>(
- new AzureDataLakeStorageProcessor({.uuid = uuid, .name = "AzureDataLakeStorageProcessor", .logger = logging::LoggerFactory<AzureDataLakeStorageProcessor>::getLogger(uuid)}, std::move(mock_data_lake_storage_client)));
+ new AzureDataLakeStorageProcessor({
+ .uuid = uuid, .name = "AzureDataLakeStorageProcessor",
+ .logger = logging::LoggerFactory<AzureDataLakeStorageProcessor>::getLogger(uuid)}, std::move(mock_data_lake_storage_client)));
auto azure_data_lake_storage_unique_ptr = std::make_unique<core::Processor>(impl->getName(), impl->getUUID(), std::move(impl));
azure_data_lake_storage_ = azure_data_lake_storage_unique_ptr.get();
auto input_dir = test_controller_.createTempDirectory();
diff --git a/extensions/azure/tests/ListAzureBlobStorageTests.cpp b/extensions/azure/tests/ListAzureBlobStorageTests.cpp
index 48c222d58..2717469ec 100644
--- a/extensions/azure/tests/ListAzureBlobStorageTests.cpp
+++ b/extensions/azure/tests/ListAzureBlobStorageTests.cpp
@@ -46,7 +46,10 @@ class ListAzureBlobStorageTestsFixture {
auto mock_blob_storage = std::make_unique<MockBlobStorage>();
mock_blob_storage_ptr_ = mock_blob_storage.get();
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
- auto impl = std::make_unique<minifi::azure::processors::ListAzureBlobStorage>(core::ProcessorMetadata{.uuid = uuid, .name = "ListAzureBlobStorage", .logger = logging::LoggerFactory<minifi::azure::processors::ListAzureBlobStorage>::getLogger(uuid)}, std::move(mock_blob_storage));
+ auto impl = std::make_unique<minifi::azure::processors::ListAzureBlobStorage>(
+ core::ProcessorMetadata{
+ .uuid = uuid, .name = "ListAzureBlobStorage",
+ .logger = logging::LoggerFactory<minifi::azure::processors::ListAzureBlobStorage>::getLogger(uuid)}, std::move(mock_blob_storage));
auto list_azure_blob_storage_unique_ptr = std::make_unique<core::Processor>(impl->getName(), impl->getUUID(), std::move(impl));
list_azure_blob_storage_ = list_azure_blob_storage_unique_ptr.get();
diff --git a/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp b/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
index 9335336f2..4cc0fa5c7 100644
--- a/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
+++ b/extensions/azure/tests/ListAzureDataLakeStorageTests.cpp
@@ -44,7 +44,9 @@ class ListAzureDataLakeStorageTestsFixture {
mock_data_lake_storage_client_ptr_ = mock_data_lake_storage_client.get();
auto uuid = utils::IdGenerator::getIdGenerator()->generate();
auto impl = std::unique_ptr<minifi::azure::processors::ListAzureDataLakeStorage>(
- new minifi::azure::processors::ListAzureDataLakeStorage({.uuid = uuid, .name = "ListAzureDataLakeStorage", .logger = logging::LoggerFactory<minifi::azure::processors::ListAzureDataLakeStorage>::getLogger(uuid)}, std::move(mock_data_lake_storage_client)));
+ new minifi::azure::processors::ListAzureDataLakeStorage({
+ .uuid = uuid, .name = "ListAzureDataLakeStorage",
+ .logger = logging::LoggerFactory<minifi::azure::processors::ListAzureDataLakeStorage>::getLogger(uuid)}, std::move(mock_data_lake_storage_client)));
auto list_azure_data_lake_storage_unique_ptr = std::make_unique<core::Processor>(impl->getName(), impl->getUUID(), std::move(impl));
list_azure_data_lake_storage_ = list_azure_data_lake_storage_unique_ptr.get();
diff --git a/extensions/pdh/PerformanceDataMonitor.cpp b/extensions/pdh/PerformanceDataMonitor.cpp
index e3ff27ad6..952c27706 100644
--- a/extensions/pdh/PerformanceDataMonitor.cpp
+++ b/extensions/pdh/PerformanceDataMonitor.cpp
@@ -65,21 +65,21 @@ void PerformanceDataMonitor::onSchedule(core::ProcessContext& context, core::Pro
void PerformanceDataMonitor::onTrigger(core::ProcessContext& context, core::ProcessSession& session) {
if (resource_consumption_counters_.empty()) {
logger_->log_error("No valid counters for PerformanceDataMonitor");
- yield();
+ context.yield();
return;
}
std::shared_ptr<core::FlowFile> flowFile = session.create();
if (!flowFile) {
logger_->log_error("Failed to create flowfile!");
- yield();
+ context.yield();
return;
}
PDH_STATUS collect_query_data_result = PdhCollectQueryData(pdh_query_);
if (ERROR_SUCCESS != collect_query_data_result) {
logger_->log_error("Error during PdhCollectQueryData, error code: {:#x}", collect_query_data_result);
- yield();
+ context.yield();
return;
}
diff --git a/libminifi/include/Funnel.h b/libminifi/include/Funnel.h
index f5a37b1df..51ad1fa12 100644
--- a/libminifi/include/Funnel.h
+++ b/libminifi/include/Funnel.h
@@ -26,7 +26,8 @@ namespace org::apache::nifi::minifi {
class Funnel final : public ForwardingNode {
public:
- Funnel(std::string_view name, const utils::Identifier& uuid) : ForwardingNode(core::ProcessorMetadata{.uuid = uuid, .name = std::string{name}, .logger = core::logging::LoggerFactory<Funnel>::getLogger(uuid)}) {}
+ Funnel(std::string_view name, const utils::Identifier& uuid)
+ : ForwardingNode(core::ProcessorMetadata{.uuid = uuid, .name = std::string{name}, .logger = core::logging::LoggerFactory<Funnel>::getLogger(uuid)}) {}
MINIFIAPI static constexpr core::annotation::Input InputRequirement = core::annotation::Input::INPUT_REQUIRED;
ADD_COMMON_VIRTUAL_FUNCTIONS_FOR_PROCESSORS
diff --git a/libminifi/include/Port.h b/libminifi/include/Port.h
index a8970e6bf..0cfd06133 100644
--- a/libminifi/include/Port.h
+++ b/libminifi/include/Port.h
@@ -34,7 +34,9 @@ class Port;
class PortImpl final : public ForwardingNode {
public:
- PortImpl(std::string_view name, const utils::Identifier& uuid, PortType port_type) : ForwardingNode({.uuid = uuid, .name = std::string{name}, .logger = core::logging::LoggerFactory<Port>::getLogger(uuid)}), port_type_(port_type) {}
+ PortImpl(std::string_view name, const utils::Identifier& uuid, PortType port_type)
+ : ForwardingNode({.uuid = uuid, .name = std::string{name}, .logger = core::logging::LoggerFactory<Port>::getLogger(uuid)}), port_type_(port_type) {}
+
PortType getPortType() const {
return port_type_;
}
diff --git a/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h b/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
index f79eb9eb7..6998e922d 100644
--- a/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
+++ b/libminifi/include/core/reporting/SiteToSiteProvenanceReportingTask.h
@@ -35,7 +35,8 @@ namespace org::apache::nifi::minifi::core::reporting {
class SiteToSiteProvenanceReportingTask : public minifi::RemoteProcessorGroupPort {
public:
explicit SiteToSiteProvenanceReportingTask(std::shared_ptr<Configure> configure)
- : minifi::RemoteProcessorGroupPort(ReportTaskName, "", std::move(configure), utils::IdGenerator::getIdGenerator()->generate(), logging::LoggerFactory<SiteToSiteProvenanceReportingTask>::getLogger()) {
+ : minifi::RemoteProcessorGroupPort(ReportTaskName, "", std::move(configure),
+ utils::IdGenerator::getIdGenerator()->generate(), logging::LoggerFactory<SiteToSiteProvenanceReportingTask>::getLogger()) {
this->setTriggerWhenEmpty(true);
batch_size_ = 100;
}
diff --git a/libminifi/src/core/ClassLoader.cpp b/libminifi/src/core/ClassLoader.cpp
index aa4be367c..ebbce7f27 100644
--- a/libminifi/src/core/ClassLoader.cpp
+++ b/libminifi/src/core/ClassLoader.cpp
@@ -134,7 +134,7 @@ class ProcessorFactoryWrapper : public ObjectFactoryImpl {
private:
std::unique_ptr<ProcessorFactory> factory_;
};
-}
+} // namespace
void ClassLoaderImpl::registerClass(const std::string &clazz, std::unique_ptr<ProcessorFactory> factory) {
registerClass(clazz, std::make_unique<ProcessorFactoryWrapper>(std::move(factory)));
diff --git a/libminifi/src/core/ConfigurableComponentImpl.cpp b/libminifi/src/core/ConfigurableComponentImpl.cpp
deleted file mode 100644
index 908c6a0e8..000000000
--- a/libminifi/src/core/ConfigurableComponentImpl.cpp
+++ /dev/null
@@ -1,149 +0,0 @@
-///**
-// *
-// * Licensed to the Apache Software Foundation (ASF) under one or more
-// * contributor license agreements. See the NOTICE file distributed with
-// * this work for additional information regarding copyright ownership.
-// * The ASF licenses this file to You under the Apache License, Version 2.0
-// * (the "License"); you may not use this file except in compliance with
-// * the License. You may obtain a copy of the License at
-// *
-// * http://www.apache.org/licenses/LICENSE-2.0
-// *
-// * Unless required by applicable law or agreed to in writing, software
-// * distributed under the License is distributed on an "AS IS" BASIS,
-// * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// * See the License for the specific language governing permissions and
-// * limitations under the License.
-// */
-//
-//#include "core/ConfigurableComponentImpl.h"
-//
-//#include "minifi-cpp/core/Property.h"
-//#include "minifi-cpp/utils/PropertyErrors.h"
-//
-//namespace org::apache::nifi::minifi::core {
-//void ConfigurableComponentImpl::setSupportedProperties(std::span<const core::PropertyReference> properties) {
-// if (!canEdit()) { return; }
-//
-// std::lock_guard<std::mutex> lock(configuration_mutex_);
-//
-// supported_properties_.clear();
-// for (const auto& item: properties) { supported_properties_.emplace(item.name, item); }
-//}
-//
-//nonstd::expected<std::string, std::error_code> ConfigurableComponentImpl::getProperty(const std::string_view name) const {
-// const std::lock_guard<std::mutex> lock(configuration_mutex_);
-// const auto it = supported_properties_.find(name);
-// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
-// const Property& prop = it->second;
-// return prop.getValue() | utils::transform([](const std::string_view value_view) -> std::string { return std::string{value_view}; });
-//}
-//
-//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::setProperty(const std::string_view name, std::string value) {
-// const std::lock_guard<std::mutex> lock(configuration_mutex_);
-// const auto it = supported_properties_.find(name);
-// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
-// Property& prop = it->second;
-//
-// return prop.setValue(std::move(value));
-//}
-//
-//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::clearProperty(const std::string_view name) {
-// const std::lock_guard<std::mutex> lock(configuration_mutex_);
-// const auto it = supported_properties_.find(name);
-// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
-// Property& prop = it->second;
-//
-// prop.clearValues();
-// return {};
-//}
-//
-//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::appendProperty(const std::string_view name, std::string value) {
-// const std::lock_guard<std::mutex> lock(configuration_mutex_);
-// const auto it = supported_properties_.find(name);
-// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
-// Property& prop = it->second;
-//
-// return prop.appendValue(std::move(value));
-//}
-//
-//nonstd::expected<std::string, std::error_code> ConfigurableComponentImpl::getDynamicProperty(const std::string_view name) const {
-// const std::lock_guard<std::mutex> lock(configuration_mutex_);
-// if (!supportsDynamicProperties()) {
-// return nonstd::make_unexpected(PropertyErrorCode::DynamicPropertiesNotSupported);
-// }
-// const auto it = dynamic_properties_.find(name);
-// if (it == dynamic_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::PropertyNotSet); }
-// const Property& prop = it->second;
-// return prop.getValue() | utils::transform([](const std::string_view value_view) -> std::string { return std::string{value_view}; });
-//}
-//
-//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::setDynamicProperty(std::string name, std::string value) {
-// const std::lock_guard<std::mutex> lock(configuration_mutex_);
-// if (!supportsDynamicProperties()) {
-// return nonstd::make_unexpected(PropertyErrorCode::DynamicPropertiesNotSupported);
-// }
-// Property& prop = dynamic_properties_[std::move(name)];
-// return prop.setValue(std::move(value));
-//}
-//
-//nonstd::expected<void, std::error_code> ConfigurableComponentImpl::appendDynamicProperty(const std::string_view name, std::string value) {
-// const std::lock_guard<std::mutex> lock(configuration_mutex_);
-// if (!supportsDynamicProperties()) {
-// return nonstd::make_unexpected(PropertyErrorCode::DynamicPropertiesNotSupported);
-// }
-// Property& prop = dynamic_properties_[std::string{name}];
-// return prop.appendValue(std::move(value));
-//}
-//
-//std::vector<std::string> ConfigurableComponentImpl::getDynamicPropertyKeys() const {
-// std::lock_guard<std::mutex> lock(configuration_mutex_);
-//
-// return dynamic_properties_ | ranges::views::transform([](const auto& kv) { return kv.first; }) | ranges::to<std::vector>();
-//}
-//
-//std::map<std::string, std::string> ConfigurableComponentImpl::getDynamicProperties() const {
-// std::lock_guard<std::mutex> lock(configuration_mutex_);
-// std::map<std::string, std::string> result;
-// for(const auto& [key,value]: dynamic_properties_) {
-// result[key] = value.getValue().value_or("");
-// }
-// return result;
-//}
-//
-//nonstd::expected<PropertyReference, std::error_code> ConfigurableComponentImpl::getPropertyReference(const std::string_view name) const {
-// const auto it = supported_properties_.find(name);
-// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
-// const Property& prop = it->second;
-// return prop.getReference();
-//}
-//
-//[[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> ConfigurableComponentImpl::getAllPropertyValues(const std::string_view name) const {
-// std::lock_guard<std::mutex> lock(configuration_mutex_);
-//
-// const auto it = supported_properties_.find(name);
-// if (it == supported_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::NotSupportedProperty); }
-// const Property& prop = it->second;
-// return prop.getAllValues() | utils::transform([](const auto& values) -> std::vector<std::string> { return std::vector<std::string>{values.begin(), values.end()}; });
-//}
-//
-//[[nodiscard]] nonstd::expected<std::vector<std::string>, std::error_code> ConfigurableComponentImpl::getAllDynamicPropertyValues(const std::string_view name) const {
-// std::lock_guard<std::mutex> lock(configuration_mutex_);
-//
-// const auto it = dynamic_properties_.find(name);
-// if (it == dynamic_properties_.end()) { return nonstd::make_unexpected(PropertyErrorCode::PropertyNotSet); }
-// const Property& prop = it->second;
-// return prop.getAllValues() | utils::transform([](const auto& values) -> std::vector<std::string> { return std::vector<std::string>{values.begin(), values.end()}; });
-//}
-//
-//[[nodiscard]] std::map<std::string, Property> ConfigurableComponentImpl::getSupportedProperties() const {
-// std::lock_guard<std::mutex> lock(configuration_mutex_);
-// std::map<std::string, Property> supported_properties;
-// for (const auto& [name, prop]: supported_properties_) {
-// supported_properties.emplace(name, prop);
-// }
-// return supported_properties;
-//
-//}
-//
-//} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/src/core/Connectable.cpp b/libminifi/src/core/Connectable.cpp
deleted file mode 100644
index 5fe3f9f07..000000000
--- a/libminifi/src/core/Connectable.cpp
+++ /dev/null
@@ -1,173 +0,0 @@
-///**
-// *
-// * Licensed to the Apache Software Foundation (ASF) under one or more
-// * contributor license agreements. See the NOTICE file distributed with
-// * this work for additional information regarding copyright ownership.
-// * The ASF licenses this file to You under the Apache License, Version 2.0
-// * (the "License"); you may not use this file except in compliance with
-// * the License. You may obtain a copy of the License at
-// *
-// * http://www.apache.org/licenses/LICENSE-2.0
-// *
-// * Unless required by applicable law or agreed to in writing, software
-// * distributed under the License is distributed on an "AS IS" BASIS,
-// * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// * See the License for the specific language governing permissions and
-// * limitations under the License.
-// */
-//#include "core/ConnectableImpl.h"
-//#include <utility>
-//#include <memory>
-//#include <string>
-//#include <set>
-//#include "core/logging/LoggerFactory.h"
-//
-//namespace org::apache::nifi::minifi::core {
-//
-//ConnectableImpl::ConnectableImpl(const std::string_view name, const utils::Identifier &uuid)
-// : CoreComponentImpl(name, uuid),
-// max_concurrent_tasks_(1),
-// connectable_version_(nullptr),
-// logger_(logging::LoggerFactory<Connectable>::getLogger(uuid_)) {
-//}
-//
-//ConnectableImpl::ConnectableImpl(const std::string_view name)
-// : CoreComponentImpl(name),
-// max_concurrent_tasks_(1),
-// connectable_version_(nullptr),
-// logger_(logging::LoggerFactory<Connectable>::getLogger(uuid_)) {
-//}
-//
-//ConnectableImpl::~ConnectableImpl() = default;
-//
-//void ConnectableImpl::setSupportedRelationships(std::span<const core::RelationshipDefinition> relationships) {
-// if (isRunning()) {
-// logger_->log_warn("Cannot set processor supported relationship while the process {} is running", name_);
-// return;
-// }
-//
-// std::lock_guard<std::mutex> lock(relationship_mutex_);
-//
-// relationships_.clear();
-// for (const auto& item : relationships) {
-// relationships_.emplace(item.name, item);
-// logger_->log_debug("Processor {} supported relationship name {}", name_, item.name);
-// }
-//}
-//
-//std::vector<Relationship> ConnectableImpl::getSupportedRelationships() const {
-// std::vector<Relationship> relationships;
-// relationships.reserve(relationships_.size());
-// for (auto const &item : relationships_) {
-// relationships.push_back(item.second);
-// }
-// return relationships;
-//}
-//
-//bool ConnectableImpl::isSupportedRelationship(const core::Relationship &relationship) {
-// // if we are running we do not need a lock since the function to change relationships_ ( setSupportedRelationships)
-// // cannot be executed while we are running
-// const bool isConnectableRunning = isRunning();
-//
-// const auto conditionalLock = isConnectableRunning ? std::unique_lock<std::mutex>() : std::unique_lock<std::mutex>(relationship_mutex_);
-//
-// return relationships_.contains(relationship.getName());
-//}
-//
-//void ConnectableImpl::addAutoTerminatedRelationship(const core::Relationship& relationship) {
-// if (isRunning()) {
-// logger_->log_warn("Can not add processor auto terminated relationship while the process {} is running", name_);
-// return;
-// }
-//
-// std::lock_guard<std::mutex> lock(relationship_mutex_);
-//
-// auto_terminated_relationships_[relationship.getName()] = relationship;
-//}
-//
-//void ConnectableImpl::setAutoTerminatedRelationships(std::span<const core::Relationship> relationships) {
-// if (isRunning()) {
-// logger_->log_warn("Can not set processor auto terminated relationship while the process {} is running", name_);
-// return;
-// }
-//
-// std::lock_guard<std::mutex> lock(relationship_mutex_);
-//
-// auto_terminated_relationships_.clear();
-// for (const auto& item : relationships) {
-// auto_terminated_relationships_[item.getName()] = item;
-// logger_->log_debug("Processor {} auto terminated relationship name {}", name_, item.getName());
-// }
-//}
-//
-//bool ConnectableImpl::isAutoTerminated(const core::Relationship &relationship) {
-// // if we are running we do not need a lock since the function to change relationships_ ( setSupportedRelationships)
-// // cannot be executed while we are running
-// const bool isConnectableRunning = isRunning();
-//
-// const auto conditionalLock = isConnectableRunning ? std::unique_lock<std::mutex>() : std::unique_lock<std::mutex>(relationship_mutex_);
-//
-// return auto_terminated_relationships_.contains(relationship.getName());
-//}
-//
-//void ConnectableImpl::waitForWork(std::chrono::milliseconds timeout) {
-// has_work_.store(isWorkAvailable());
-//
-// if (!has_work_.load()) {
-// std::unique_lock<std::mutex> lock(work_available_mutex_);
-// work_condition_.wait_for(lock, timeout, [&] {return has_work_.load();});
-// }
-//}
-//
-//void ConnectableImpl::notifyWork() {
-// // Do nothing if we are not event-driven
-// if (strategy_ != EVENT_DRIVEN) {
-// return;
-// }
-//
-// {
-// has_work_.store(isWorkAvailable());
-//
-// if (has_work_.load()) {
-// work_condition_.notify_one();
-// }
-// }
-//}
-//
-//std::set<Connectable*> ConnectableImpl::getOutGoingConnections(const std::string &relationship) {
-// const auto it = outgoing_connections_.find(relationship);
-// if (it != outgoing_connections_.end()) {
-// return it->second;
-// } else if (relationship == "__self__") {
-// return {this};
-// } else {
-// return {};
-// }
-//}
-//
-//Connectable* ConnectableImpl::getNextIncomingConnection() {
-// std::lock_guard<std::mutex> lock(relationship_mutex_);
-// return getNextIncomingConnectionImpl(lock);
-//}
-//
-//Connectable* ConnectableImpl::getNextIncomingConnectionImpl(const std::lock_guard<std::mutex>& /*relatioship_mutex_lock*/) {
-// if (incoming_connections_.empty())
-// return nullptr;
-//
-// if (incoming_connections_Iter == incoming_connections_.end())
-// incoming_connections_Iter = incoming_connections_.begin();
-//
-// auto ret = *incoming_connections_Iter;
-// incoming_connections_Iter++;
-//
-// if (incoming_connections_Iter == incoming_connections_.end())
-// incoming_connections_Iter = incoming_connections_.begin();
-//
-// return ret;
-//}
-//
-//Connectable* ConnectableImpl::pickIncomingConnection() {
-// return getNextIncomingConnection();
-//}
-//
-//} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/src/core/Processor.cpp b/libminifi/src/core/Processor.cpp
index a70867ce7..d1188c234 100644
--- a/libminifi/src/core/Processor.cpp
+++ b/libminifi/src/core/Processor.cpp
@@ -38,7 +38,6 @@
#include "range/v3/algorithm/any_of.hpp"
#include "fmt/format.h"
#include "Exception.h"
-#include "core/Processor.h"
#include "core/ProcessorMetrics.h"
using namespace std::literals::chrono_literals;
diff --git a/libminifi/test/libtest/unit/DummyProcessor.h b/libminifi/test/libtest/unit/DummyProcessor.h
index 0e68bdfb2..ba7f7bc08 100644
--- a/libminifi/test/libtest/unit/DummyProcessor.h
+++ b/libminifi/test/libtest/unit/DummyProcessor.h
@@ -30,7 +30,7 @@ class DummyProcessor : public minifi::core::ProcessorImpl {
public:
using ProcessorImpl::ProcessorImpl;
- DummyProcessor(std::string_view name)
+ explicit DummyProcessor(std::string_view name)
: ProcessorImpl{minifi::core::ProcessorMetadata{
.uuid = minifi::utils::IdGenerator::getIdGenerator()->generate(),
.name = std::string{name},
diff --git a/libminifi/test/libtest/unit/ProcessorUtils.h b/libminifi/test/libtest/unit/ProcessorUtils.h
index 36a270d9d..dd637c7dd 100644
--- a/libminifi/test/libtest/unit/ProcessorUtils.h
+++ b/libminifi/test/libtest/unit/ProcessorUtils.h
@@ -43,4 +43,4 @@ std::unique_ptr<core::Processor> make_custom_processor(Args&&... args) {
return std::make_unique<core::Processor>(name, uuid, std::move(processor_impl));
}
-} // namespace org::apache::nifi::minifi::test::utils
\ No newline at end of file
+} // namespace org::apache::nifi::minifi::test::utils
diff --git a/libminifi/test/libtest/unit/TestBase.h b/libminifi/test/libtest/unit/TestBase.h
index 67a155675..8ecef5ab6 100644
--- a/libminifi/test/libtest/unit/TestBase.h
+++ b/libminifi/test/libtest/unit/TestBase.h
@@ -205,7 +205,7 @@ template<typename T>
class TypedProcessorWrapper {
public:
TypedProcessorWrapper() = default;
- TypedProcessorWrapper(core::Processor* proc): proc_(proc) {
+ TypedProcessorWrapper(core::Processor* proc): proc_(proc) { // NOLINT(runtime/explicit)
if (proc_) {
proc_->getImpl<T>();
}
(nifi-minifi-cpp) 27/37: Provide the correct name on processor construction
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit b4645cbb4e042853b99abf66cc5490cab6b972cb
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Jul 3 10:32:14 2025 +0200
Provide the correct name on processor construction
---
core-framework/include/core/ClassLoader.h | 6 ++++++
libminifi/include/core/FlowConfiguration.h | 3 +--
libminifi/include/processors/ProcessorUtils.h | 6 +++---
libminifi/src/core/ClassLoader.cpp | 20 ++++++++++++++++++++
libminifi/src/core/FlowConfiguration.cpp | 15 +++------------
libminifi/src/core/flow/StructuredConfiguration.cpp | 2 +-
minifi-api/include/minifi-cpp/core/ClassLoader.h | 5 +++++
7 files changed, 39 insertions(+), 18 deletions(-)
diff --git a/core-framework/include/core/ClassLoader.h b/core-framework/include/core/ClassLoader.h
index fc6cba043..26915db5b 100644
--- a/core-framework/include/core/ClassLoader.h
+++ b/core-framework/include/core/ClassLoader.h
@@ -44,5 +44,11 @@ T *ClassLoader::instantiateRaw(const std::string &class_name, const std::string
}));
}
+template<class T>
+std::unique_ptr<T> ClassLoader::instantiate(const std::string &class_name, const std::string &name, const utils::Identifier &uuid) {
+ return utils::dynamic_unique_cast<T>(instantiate(class_name, name, uuid, [] (CoreComponent* obj) -> bool {
+ return dynamic_cast<T*>(obj) != nullptr;
+ }));
+}
} // namespace org::apache::nifi::minifi::core
diff --git a/libminifi/include/core/FlowConfiguration.h b/libminifi/include/core/FlowConfiguration.h
index 1145601f2..8d83efcaf 100644
--- a/libminifi/include/core/FlowConfiguration.h
+++ b/libminifi/include/core/FlowConfiguration.h
@@ -84,8 +84,7 @@ class FlowConfiguration : public CoreComponentImpl {
~FlowConfiguration() override;
// Create Processor (Node/Input/Output Port) based on the name
- std::unique_ptr<core::Processor> createProcessor(const std::string &name, const utils::Identifier &uuid);
- std::unique_ptr<core::Processor> createProcessor(const std::string &name, const std::string &fullname, const utils::Identifier &uuid);
+ std::unique_ptr<core::Processor> createProcessor(const std::string &class_short, const std::string &fullclass, const std::string &name, const utils::Identifier &uuid);
// Create Root Processor Group
static std::unique_ptr<core::ProcessGroup> createRootProcessGroup(const std::string &name, const utils::Identifier &uuid, int version);
diff --git a/libminifi/include/processors/ProcessorUtils.h b/libminifi/include/processors/ProcessorUtils.h
index fe4c8fc8a..7f5452e78 100644
--- a/libminifi/include/processors/ProcessorUtils.h
+++ b/libminifi/include/processors/ProcessorUtils.h
@@ -36,10 +36,10 @@ class ProcessorUtils {
* @param canonicalName full class name ( canonical name )
* @param uuid uuid object for the processor
*/
- static inline std::unique_ptr<core::Processor> createProcessor(const std::string &class_short_name, const std::string &canonicalName, const utils::Identifier &uuid) {
- auto ptr = core::ClassLoader::getDefaultClassLoader().instantiate(class_short_name, uuid);
+ static inline std::unique_ptr<core::Processor> createProcessor(const std::string &class_short_name, const std::string &canonicalName, const std::string &name, const utils::Identifier &uuid) {
+ auto ptr = core::ClassLoader::getDefaultClassLoader().instantiate(class_short_name, name, uuid);
if (ptr == nullptr) {
- ptr = core::ClassLoader::getDefaultClassLoader().instantiate(canonicalName, uuid);
+ ptr = core::ClassLoader::getDefaultClassLoader().instantiate(canonicalName, name, uuid);
}
if (ptr == nullptr) {
return nullptr;
diff --git a/libminifi/src/core/ClassLoader.cpp b/libminifi/src/core/ClassLoader.cpp
index b7076b369..c46c75e36 100644
--- a/libminifi/src/core/ClassLoader.cpp
+++ b/libminifi/src/core/ClassLoader.cpp
@@ -50,6 +50,8 @@ class ClassLoaderImpl : public ClassLoader {
[[nodiscard]] std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) override;
+ [[nodiscard]] std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const std::string &name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) override;
+
[[nodiscard]] CoreComponent* instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) override;
~ClassLoaderImpl() override = default;
@@ -200,6 +202,24 @@ std::unique_ptr<CoreComponent> ClassLoaderImpl::instantiate(const std::string &c
return nullptr;
}
+std::unique_ptr<CoreComponent> ClassLoaderImpl::instantiate(const std::string &class_name, const std::string &name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) {
+ std::lock_guard<std::mutex> lock(internal_mutex_);
+ // allow subsequent classes to override functionality (like ProcessContextBuilder)
+ for (auto& child_loader : class_loaders_) {
+ if (auto result = child_loader.second.instantiate(class_name, name, uuid, filter)) {
+ return result;
+ }
+ }
+ auto factory_entry = loaded_factories_.find(class_name);
+ if (factory_entry != loaded_factories_.end()) {
+ auto obj = factory_entry->second->create(name, uuid);
+ if (filter(obj.get())) {
+ return obj;
+ }
+ }
+ return nullptr;
+}
+
CoreComponent* ClassLoaderImpl::instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) {
std::lock_guard<std::mutex> lock(internal_mutex_);
// allow subsequent classes to override functionality (like ProcessContextBuilder)
diff --git a/libminifi/src/core/FlowConfiguration.cpp b/libminifi/src/core/FlowConfiguration.cpp
index 2e1659dc8..d2c21ee9c 100644
--- a/libminifi/src/core/FlowConfiguration.cpp
+++ b/libminifi/src/core/FlowConfiguration.cpp
@@ -71,19 +71,10 @@ FlowConfiguration::~FlowConfiguration() {
}
}
-std::unique_ptr<core::Processor> FlowConfiguration::createProcessor(const std::string &name, const utils::Identifier &uuid) {
- auto processor = minifi::processors::ProcessorUtils::createProcessor(name, name, uuid);
+std::unique_ptr<core::Processor> FlowConfiguration::createProcessor(const std::string &class_short, const std::string &fullclass, const std::string &name, const utils::Identifier &uuid) {
+ auto processor = minifi::processors::ProcessorUtils::createProcessor(class_short, fullclass, name, uuid);
if (nullptr == processor) {
- logger_->log_error("No Processor defined for {}", name);
- return nullptr;
- }
- return processor;
-}
-
-std::unique_ptr<core::Processor> FlowConfiguration::createProcessor(const std::string &name, const std::string &fullname, const utils::Identifier &uuid) {
- auto processor = minifi::processors::ProcessorUtils::createProcessor(name, fullname, uuid);
- if (nullptr == processor) {
- logger_->log_error("No Processor defined for {}", fullname);
+ logger_->log_error("No Processor defined for {}", fullclass);
return nullptr;
}
return processor;
diff --git a/libminifi/src/core/flow/StructuredConfiguration.cpp b/libminifi/src/core/flow/StructuredConfiguration.cpp
index 667aa3205..55411616d 100644
--- a/libminifi/src/core/flow/StructuredConfiguration.cpp
+++ b/libminifi/src/core/flow/StructuredConfiguration.cpp
@@ -328,7 +328,7 @@ void StructuredConfiguration::parseProcessorNode(const Node& processors_node, co
logger_->log_debug("parseProcessorNode: class => [{}]", procCfg.javaClass);
// Determine the processor name only from the Java class
- processor = createProcessor(utils::string::partAfterLastOccurrenceOf(procCfg.javaClass, '.'), procCfg.javaClass, uuid);
+ processor = createProcessor(utils::string::partAfterLastOccurrenceOf(procCfg.javaClass, '.'), procCfg.javaClass, procCfg.name, uuid);
if (!processor) {
logger_->log_error("Could not create a processor {} with id {}", procCfg.name, procCfg.id);
throw std::invalid_argument("Could not create processor " + procCfg.name);
diff --git a/minifi-api/include/minifi-cpp/core/ClassLoader.h b/minifi-api/include/minifi-cpp/core/ClassLoader.h
index 72fa03319..56e031f39 100644
--- a/minifi-api/include/minifi-cpp/core/ClassLoader.h
+++ b/minifi-api/include/minifi-cpp/core/ClassLoader.h
@@ -75,6 +75,8 @@ class ClassLoader {
[[nodiscard]] virtual std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) = 0;
+ [[nodiscard]] virtual std::unique_ptr<CoreComponent> instantiate(const std::string &class_name, const std::string &name, const utils::Identifier &uuid, std::function<bool(CoreComponent*)> filter) = 0;
+
[[nodiscard]] virtual CoreComponent* instantiateRaw(const std::string &class_name, const std::string &name, std::function<bool(CoreComponent*)> filter) = 0;
/**
@@ -103,6 +105,9 @@ class ClassLoader {
*/
template<class T = CoreComponent>
[[nodiscard]] T *instantiateRaw(const std::string &class_name, const std::string &name);
+
+ template<class T = CoreComponent>
+ [[nodiscard]] std::unique_ptr<T> instantiate(const std::string &class_name, const std::string &name, const utils::Identifier &uuid);
};
} // namespace org::apache::nifi::minifi::core
(nifi-minifi-cpp) 34/37: Linter fix
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit a96d25102b0dd4d7a86c12d3be414ff6422836e8
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Mon Jul 21 14:00:28 2025 +0200
Linter fix
---
minifi-api/include/minifi-cpp/core/ProcessorMetadata.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/minifi-api/include/minifi-cpp/core/ProcessorMetadata.h b/minifi-api/include/minifi-cpp/core/ProcessorMetadata.h
index b40e51faf..909ba731d 100644
--- a/minifi-api/include/minifi-cpp/core/ProcessorMetadata.h
+++ b/minifi-api/include/minifi-cpp/core/ProcessorMetadata.h
@@ -16,10 +16,10 @@
*/
#pragma once
-#include "minifi-cpp/utils/Id.h"
-#include "minifi-cpp/core/logging/Logger.h"
#include <memory>
#include <string>
+#include "minifi-cpp/utils/Id.h"
+#include "minifi-cpp/core/logging/Logger.h"
namespace org::apache::nifi::minifi::core {
(nifi-minifi-cpp) 06/37: Trigger ci
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adebreceni pushed a commit to branch minifi-api-reduced
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
commit b7d3b52fe614276ef03f9598ca7221aaafd24265
Author: Adam Debreceni <ad...@apache.org>
AuthorDate: Thu Apr 10 08:34:07 2025 +0200
Trigger ci