You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "lordgamez (via GitHub)" <gi...@apache.org> on 2026/02/18 16:26:35 UTC
[PR] MINIFICPP-2644 Add proxy controller service support for GCP processors [nifi-minifi-cpp]
lordgamez opened a new pull request, #2112:
URL: https://github.com/apache/nifi-minifi-cpp/pull/2112
https://issues.apache.org/jira/browse/MINIFICPP-2644
Depends on https://issues.apache.org/jira/browse/MINIFICPP-2644
--------------
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically main)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINIFICPP-2644 Add proxy controller service support for GCP processors [nifi-minifi-cpp]
Posted by "szaszm (via GitHub)" <gi...@apache.org>.
szaszm commented on code in PR #2112:
URL: https://github.com/apache/nifi-minifi-cpp/pull/2112#discussion_r3167557854
##########
extensions/gcp/processors/GCSProcessor.cpp:
##########
@@ -48,13 +48,46 @@ void GCSProcessor::onSchedule(core::ProcessContext& context, core::ProcessSessio
endpoint_url_ = context.getProperty(EndpointOverrideURL) | utils::toOptional();
if (endpoint_url_)
logger_->log_debug("Endpoint overwritten: {}", *endpoint_url_);
+
+ auto proxy_controller_service = minifi::utils::parseOptionalControllerService<minifi::controllers::ProxyConfigurationServiceInterface>(context, ProxyConfigurationService, getUUID());
+ if (proxy_controller_service) {
+ if (proxy_controller_service->getProxyType() == minifi::controllers::ProxyType::DIRECT) {
+ logger_->log_debug("Proxy configuration service indicates DIRECT connection, no proxy will be used for GCS processor");
+ return;
+ }
Review Comment:
This `return` seems error-prone to me in the middle of a function that has more than one responsibility. I can imagine adding another property and its reading to the end of onSchedule later, and it would accidentally get skipped when the ProxyControllerService is configured to DIRECT.
```suggestion
if (proxy_controller_service && proxy_controller_service->getProxyType() != minifi::controllers::ProxyType::DIRECT) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINIFICPP-2644 Add proxy controller service support for GCP processors [nifi-minifi-cpp]
Posted by "lordgamez (via GitHub)" <gi...@apache.org>.
lordgamez commented on code in PR #2112:
URL: https://github.com/apache/nifi-minifi-cpp/pull/2112#discussion_r3168782247
##########
extensions/gcp/processors/GCSProcessor.cpp:
##########
@@ -48,13 +48,46 @@ void GCSProcessor::onSchedule(core::ProcessContext& context, core::ProcessSessio
endpoint_url_ = context.getProperty(EndpointOverrideURL) | utils::toOptional();
if (endpoint_url_)
logger_->log_debug("Endpoint overwritten: {}", *endpoint_url_);
+
+ auto proxy_controller_service = minifi::utils::parseOptionalControllerService<minifi::controllers::ProxyConfigurationServiceInterface>(context, ProxyConfigurationService, getUUID());
+ if (proxy_controller_service) {
+ if (proxy_controller_service->getProxyType() == minifi::controllers::ProxyType::DIRECT) {
+ logger_->log_debug("Proxy configuration service indicates DIRECT connection, no proxy will be used for GCS processor");
+ return;
+ }
Review Comment:
Updated in https://github.com/apache/nifi-minifi-cpp/pull/2112/commits/5eae350c864470a83956c83fd6c6d04cac8207ad
##########
extensions/gcp/processors/GCSProcessor.cpp:
##########
@@ -48,13 +48,42 @@ void GCSProcessor::onSchedule(core::ProcessContext& context, core::ProcessSessio
endpoint_url_ = context.getProperty(EndpointOverrideURL) | utils::toOptional();
if (endpoint_url_)
logger_->log_debug("Endpoint overwritten: {}", *endpoint_url_);
+
+ auto proxy_controller_service = minifi::utils::parseOptionalControllerService<minifi::controllers::ProxyConfigurationServiceInterface>(context, ProxyConfigurationService, getUUID());
+ if (proxy_controller_service) {
+ if (proxy_controller_service->getProxyType() == minifi::controllers::ProxyType::DIRECT) {
+ logger_->log_debug("Proxy configuration service indicates DIRECT connection, no proxy will be used for GCS processor");
+ return;
+ }
+
+ logger_->log_debug("Proxy configuration is set for GCS processor");
+
+ proxy_ = google::cloud::ProxyConfig{};
+ proxy_->set_scheme(minifi::utils::string::startsWith(proxy_controller_service->getHost(), "https") ? "https" : "http");
+ auto proxy_host = proxy_controller_service->getHost();
+ if (minifi::utils::string::startsWith(proxy_host, "https://")) {
+ proxy_host = proxy_host.substr(8);
+ } else if (minifi::utils::string::startsWith(proxy_host, "http://")) {
+ proxy_host = proxy_host.substr(7);
Review Comment:
Updated in https://github.com/apache/nifi-minifi-cpp/pull/2112/commits/5eae350c864470a83956c83fd6c6d04cac8207ad
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINIFICPP-2644 Add proxy controller service support for GCP processors [nifi-minifi-cpp]
Posted by "martinzink (via GitHub)" <gi...@apache.org>.
martinzink merged PR #2112:
URL: https://github.com/apache/nifi-minifi-cpp/pull/2112
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINIFICPP-2644 Add proxy controller service support for GCP processors [nifi-minifi-cpp]
Posted by "szaszm (via GitHub)" <gi...@apache.org>.
szaszm commented on code in PR #2112:
URL: https://github.com/apache/nifi-minifi-cpp/pull/2112#discussion_r3167600719
##########
extensions/gcp/processors/GCSProcessor.cpp:
##########
@@ -48,13 +48,42 @@ void GCSProcessor::onSchedule(core::ProcessContext& context, core::ProcessSessio
endpoint_url_ = context.getProperty(EndpointOverrideURL) | utils::toOptional();
if (endpoint_url_)
logger_->log_debug("Endpoint overwritten: {}", *endpoint_url_);
+
+ auto proxy_controller_service = minifi::utils::parseOptionalControllerService<minifi::controllers::ProxyConfigurationServiceInterface>(context, ProxyConfigurationService, getUUID());
+ if (proxy_controller_service) {
+ if (proxy_controller_service->getProxyType() == minifi::controllers::ProxyType::DIRECT) {
+ logger_->log_debug("Proxy configuration service indicates DIRECT connection, no proxy will be used for GCS processor");
+ return;
+ }
+
+ logger_->log_debug("Proxy configuration is set for GCS processor");
+
+ proxy_ = google::cloud::ProxyConfig{};
+ proxy_->set_scheme(minifi::utils::string::startsWith(proxy_controller_service->getHost(), "https") ? "https" : "http");
+ auto proxy_host = proxy_controller_service->getHost();
+ if (minifi::utils::string::startsWith(proxy_host, "https://")) {
+ proxy_host = proxy_host.substr(8);
+ } else if (minifi::utils::string::startsWith(proxy_host, "http://")) {
+ proxy_host = proxy_host.substr(7);
Review Comment:
I think it would be more future-proof/less error-prone to directly connect the string prefixes and their lengths by introducing symbolical constants
```suggestion
constexpr std::string_view https_prefix = "https://";
constexpr std::string_view http_prefix = "http://";
if (minifi::utils::string::startsWith(proxy_host, https_prefix)) {
proxy_host = proxy_host.substr(https_prefix.size());
} else if (minifi::utils::string::startsWith(proxy_host, http_prefix)) {
proxy_host = proxy_host.substr(http_prefix.size());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINIFICPP-2644 Add proxy controller service support for GCP processors [nifi-minifi-cpp]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #2112:
URL: https://github.com/apache/nifi-minifi-cpp/pull/2112#discussion_r2846925260
##########
extensions/gcp/processors/GCSProcessor.h:
##########
@@ -53,10 +55,16 @@ class GCSProcessor : public core::ProcessorImpl {
.isRequired(false)
.supportsExpressionLanguage(true)
.build();
+ EXTENSIONAPI static constexpr auto ProxyConfigurationService = core::PropertyDefinitionBuilder<>::createProperty("Proxy Configuration Service")
+ .withDescription("Specifies the Proxy Configuration Controller Service to proxy network requests. When used, "
+ "this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties.")
Review Comment:
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. Consider updating the description to remove the reference to individual proxy properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
```suggestion
.withDescription("Specifies the Proxy Configuration Controller Service to proxy network requests.")
```
##########
PROCESSORS.md:
##########
@@ -2447,19 +2450,20 @@ Puts flow files to a Google Cloud Storage Bucket.
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-|--------------------------------------|---------------|------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. |
-| **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. |
-| Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** |
-| Bucket | ${gcs.bucket} | | Bucket of the object.<br/>**Supports Expression Language: true** |
-| Key | ${filename} | | Name of the object.<br/>**Supports Expression Language: true** |
-| Content Type | ${mime.type} | | Content Type for the file, i.e. text/plain <br/>**Supports Expression Language: true** |
-| MD5 Hash | | | MD5 Hash (encoded in Base64) of the file for server-side validation.<br/>**Supports Expression Language: true** |
-| CRC32C Checksum | | | CRC32C Checksum (encoded in Base64, big-Endian order) of the file for server-side validation.<br/>**Supports Expression Language: true** |
-| Server Side Encryption Key | | | An AES256 Encryption Key (encoded in base64) for server-side encryption of the object.<br/>**Sensitive Property: true**<br/>**Supports Expression Language: true** |
-| Object ACL | | authenticatedRead<br/>bucketOwnerFullControl<br/>bucketOwnerRead<br/>private<br/>projectPrivate<br/>publicRead<br/>publicReadWrite | Access Control to be attached to the object uploaded. Not providing this will revert to bucket defaults. |
-| Overwrite Object | true | true<br/>false | If false, the upload to GCS will succeed only if the object does not exist. |
+| Name | Default Value | Allowable Values | Description |
+|--------------------------------------|---------------|------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. |
+| **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. |
+| Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** |
+| Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. |
Review Comment:
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. The description should be updated to remove the reference to non-existent properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
```suggestion
| Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. |
```
##########
PROCESSORS.md:
##########
@@ -609,15 +609,16 @@ Deletes an object from a Google Cloud Bucket.
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-|--------------------------------------|---------------|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------|
-| **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. |
-| **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. |
-| Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** |
-| Bucket | ${gcs.bucket} | | Bucket of the object.<br/>**Supports Expression Language: true** |
-| Key | ${filename} | | Name of the object.<br/>**Supports Expression Language: true** |
-| Server Side Encryption Key | | | The AES256 Encryption Key (encoded in base64) for server-side decryption of the object.<br/>**Supports Expression Language: true** |
-| Object Generation | | | The generation of the Object to download. If left empty, then it will download the latest generation.<br/>**Supports Expression Language: true** |
+| Name | Default Value | Allowable Values | Description |
+|--------------------------------------|---------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. |
+| **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. |
+| Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** |
+| Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. |
Review Comment:
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. The description should be updated to remove the reference to non-existent properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
```suggestion
| Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. |
```
##########
extensions/gcp/processors/GCSProcessor.h:
##########
@@ -30,6 +30,8 @@
#include "google/cloud/credentials.h"
#include "google/cloud/storage/client.h"
#include "google/cloud/storage/retry_policy.h"
+#include "minifi-cpp/controllers/ProxyConfigurationServiceInterface.h"
+#include "controllers/ProxyConfiguration.h"
Review Comment:
The include "controllers/ProxyConfiguration.h" is not needed in this header file. The header only uses google::cloud::ProxyConfig (line 79), not minifi::controllers::ProxyConfiguration. This include should be removed to reduce unnecessary dependencies and improve compilation times.
```suggestion
```
##########
PROCESSORS.md:
##########
@@ -1632,13 +1634,14 @@ Retrieves a listing of objects from an GCS bucket. For each object that is liste
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-|--------------------------------------|---------------|------------------|-----------------------------------------------------------------------------------------------------------------------------------|
-| **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. |
-| **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. |
-| Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** |
-| **Bucket** | | | Bucket of the object.<br/>**Supports Expression Language: true** |
-| List all versions | false | true<br/>false | Set this option to `true` to get all the previous versions separately. |
+| Name | Default Value | Allowable Values | Description |
+|--------------------------------------|---------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. |
+| **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. |
+| Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** |
+| Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. |
Review Comment:
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. The description should be updated to remove the reference to non-existent properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
```suggestion
| Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. |
```
##########
PROCESSORS.md:
##########
@@ -909,15 +910,16 @@ Fetches a file from a Google Cloud Bucket. Designed to be used in tandem with Li
In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.
-| Name | Default Value | Allowable Values | Description |
-|--------------------------------------|---------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. |
-| **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. |
-| Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** |
-| Bucket | ${gcs.bucket} | | Bucket of the object.<br/>**Supports Expression Language: true** |
-| Key | ${filename} | | Name of the object.<br/>**Supports Expression Language: true** |
-| Server Side Encryption Key | | | The AES256 Encryption Key (encoded in base64) for server-side decryption of the object.<br/>**Sensitive Property: true**<br/>**Supports Expression Language: true** |
-| Object Generation | | | The generation of the Object to download. If left empty, then it will download the latest generation.<br/>**Supports Expression Language: true** |
+| Name | Default Value | Allowable Values | Description |
+|--------------------------------------|---------------|------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **GCP Credentials Provider Service** | | | The Controller Service used to obtain Google Cloud Platform credentials. Should be the name of a GCPCredentialsControllerService. |
+| **Number of retries** | 6 | | How many retry attempts should be made before routing to the failure relationship. |
+| Endpoint Override URL | | | Overrides the default Google Cloud Storage endpoints<br/>**Supports Expression Language: true** |
+| Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. When used, this will override any values specified for Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties. |
Review Comment:
The description for the Proxy Configuration Service property mentions overriding "Proxy Host, Proxy Port, Proxy Username, and Proxy Password properties," but GCP processors do not have these individual proxy properties. Unlike AWS processors which support both individual proxy properties and a proxy configuration service, GCP processors only support the proxy configuration service. The description should be updated to remove the reference to non-existent properties, or simply state: "Specifies the Proxy Configuration Controller Service to proxy network requests."
```suggestion
| Proxy Configuration Service | | | Specifies the Proxy Configuration Controller Service to proxy network requests. |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org