You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by mans2singh <gi...@git.apache.org> on 2016/02/08 20:59:15 UTC
[GitHub] nifi pull request: Added support for using proxy for aws processor...
GitHub user mans2singh opened a pull request:
https://github.com/apache/nifi/pull/209
Added support for using proxy for aws processors
Added http proxy support for aws processors
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mans2singh/nifi nifi-1489
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/209.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #209
----
commit 83c05145ffe9caab932698cb970cde27d905cf81
Author: mans2singh <ma...@yahoo.com>
Date: 2016-02-08T19:43:03Z
Added support for using proxy
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by pljplj <gi...@git.apache.org>.
Github user pljplj commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-191278133
Is there a version of the aws proxy that is compatible with 0.5.1 ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by mans2singh <gi...@git.apache.org>.
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-182092294
Explicitly check that proxy host and port are both specified based on comments (rather than using xor operator)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by mans2singh <gi...@git.apache.org>.
Github user mans2singh commented on a diff in the pull request:
https://github.com/apache/nifi/pull/209#discussion_r53062954
--- Diff: nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java ---
@@ -91,6 +93,23 @@
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.sensitive(true)
.build();
+
+ public static final PropertyDescriptor PROXY_HOST = new PropertyDescriptor.Builder()
+ .name("Proxy Host")
+ .description("Proxy host name or ip")
+ .expressionLanguageSupported(true)
+ .required(false)
+ .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+ .build();
+
+ public static final PropertyDescriptor PROXY_HOST_PORT = new PropertyDescriptor.Builder()
+ .name("Proxy Host Port")
+ .description("Proxy host port (must be postive integer)")
+ .expressionLanguageSupported(true)
+ .required(false)
+ .addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
--- End diff --
@apiri - You are absolutely right on that. Please go ahead and let me know if there is anything else required from me.
Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on a diff in the pull request:
https://github.com/apache/nifi/pull/209#discussion_r53044987
--- Diff: nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java ---
@@ -91,6 +93,23 @@
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.sensitive(true)
.build();
+
+ public static final PropertyDescriptor PROXY_HOST = new PropertyDescriptor.Builder()
+ .name("Proxy Host")
+ .description("Proxy host name or ip")
+ .expressionLanguageSupported(true)
+ .required(false)
+ .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+ .build();
+
+ public static final PropertyDescriptor PROXY_HOST_PORT = new PropertyDescriptor.Builder()
+ .name("Proxy Host Port")
+ .description("Proxy host port (must be postive integer)")
+ .expressionLanguageSupported(true)
+ .required(false)
+ .addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
--- End diff --
@mans2singh One quick note, do you mind if I convert this validator to the StandardValidators.PORT_VALIDATOR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-184861933
@mans2singh Yep, saw it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-191360199
@pljplj Not sure about an alternative means of using the proxy, but this particular patch is scheduled for 0.6.0 given its feature bearing nature.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by apiri <gi...@git.apache.org>.
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-184782171
@mans2singh @jskora
Was able to test against S3 using squid proxy and verify this worked as intended. Will get merged into master. Thanks for the contribution!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] nifi pull request: Added support for using proxy for
aws processor...
Posted by M Singh <ma...@yahoo.com.INVALID>.
Hi:
I've corrected the logic based on your comments.
Thanks
Mans
On Tuesday, February 9, 2016 11:46 AM, jskora <gi...@git.apache.org> wrote:
Github user jskora commented on a diff in the pull request:
https://github.com/apache/nifi/pull/209#discussion_r52362955
--- Diff: nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java ---
@@ -161,6 +180,12 @@ private static AllowableValue createAllowableValue(final Regions regions) {
problems.add(new ValidationResult.Builder().input("Access Key").valid(false).explanation("Cannot set both Credentials File and Secret Key/Access Key").build());
}
+ final boolean proxyHostSet = validationContext.getProperty(PROXY_HOST).isSet();
+ final boolean proxyHostPortSet = validationContext.getProperty(PROXY_HOST_PORT).isSet();
+ if (proxyHostSet ^ proxyHostPortSet ) {
--- End diff --
The [Java operators](https://docs.oracle.com/javase/tutorial/java/nutsandbolts/opsummary.html) consider the XOR operator ("^") to be a bitwise operator not a conditional logic operator. Even if it works correctly on this line, I think it would be more clear and less likely to be misinterpreted if the logical expressions were spelled out explicitly.
` if ((proxyHostSet && !proxyHostPortSet) || (!proxyHostSet && proxyHostPortSet)) {`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by jskora <gi...@git.apache.org>.
Github user jskora commented on a diff in the pull request:
https://github.com/apache/nifi/pull/209#discussion_r52362955
--- Diff: nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java ---
@@ -161,6 +180,12 @@ private static AllowableValue createAllowableValue(final Regions regions) {
problems.add(new ValidationResult.Builder().input("Access Key").valid(false).explanation("Cannot set both Credentials File and Secret Key/Access Key").build());
}
+ final boolean proxyHostSet = validationContext.getProperty(PROXY_HOST).isSet();
+ final boolean proxyHostPortSet = validationContext.getProperty(PROXY_HOST_PORT).isSet();
+ if (proxyHostSet ^ proxyHostPortSet ) {
--- End diff --
The [Java operators](https://docs.oracle.com/javase/tutorial/java/nutsandbolts/opsummary.html) consider the XOR operator ("^") to be a bitwise operator not a conditional logic operator. Even if it works correctly on this line, I think it would be more clear and less likely to be misinterpreted if the logical expressions were spelled out explicitly.
` if ((proxyHostSet && !proxyHostPortSet) || (!proxyHostSet && proxyHostPortSet)) {`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by mans2singh <gi...@git.apache.org>.
Github user mans2singh commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-184858822
BTW - I had a combined request for kinesis + proxy - if you would like to review - https://github.com/apache/nifi/pull/213
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by jskora <gi...@git.apache.org>.
Github user jskora commented on the pull request:
https://github.com/apache/nifi/pull/209#issuecomment-182105514
Looks good based on the code. I can't test it today, but I might be able to later this week.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: Added support for using proxy for aws processor...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/209
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---