You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2020/05/15 23:49:08 UTC
[GitHub] [nifi] matthew-formosa-gig opened a new pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
matthew-formosa-gig opened a new pull request #4278:
URL: https://github.com/apache/nifi/pull/4278
#### Description of PR
The current implementation of the Ignite processors forces one to make use of the Thick Ignite client. This may be problematic in cases where Ignite resides within a Kubernetes cluster while NiFi is outside of the Kubernetes cluster. Also, it may be the case that for certain scenarios the thin Ignite client might be a better fit than the thick Ignite client.
With this feature the possibility of choosing between the two clients will be allowed.
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target branch (typically `master`)?
- [X] Is your initial contribution a single, squashed commit?
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [X] Have you verified that the full build is successful on both JDK 8 and JDK 11?
- [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [X] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] joewitt commented on a change in pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
joewitt commented on a change in pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#discussion_r426155221
##########
File path: nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/test/resources/test-default-ignite-client.xml
##########
@@ -41,4 +37,12 @@
</bean>
</property>
</bean>
+
+ <bean abstract="true" id="thinIgniteClient" class="org.apache.ignite.configuration.ClientConfiguration">
+ <property name="addresses">
+ <list>
+ <value>127.0.0.1:10800</value>
Review comment:
having statically defined ports in tests almost always is a recipe for spurious failures. if any other test is running at this time on this port then it will fail. I'm not sure if there is an option to adjust this but if not we need to ensure none of these tests actually use this port as unit tests - only integration tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
joewitt commented on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-629681317
cool. nice work on this - i'll take a look soon hopefully. want to verify the dependency set/licensing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] pvillard31 commented on a change in pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
pvillard31 commented on a change in pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#discussion_r636184907
##########
File path: nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/pom.xml
##########
@@ -24,23 +25,11 @@
<artifactId>nifi-ignite-processors</artifactId>
<packaging>jar</packaging>
+ <properties>
+ <ignite.version>2.8.0</ignite.version>
Review comment:
```suggestion
<ignite.version>2.10.0</ignite.version>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
pvillard31 commented on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-845339872
Hi @matthew-formosa-gig - I know this is an old one, but happy to spend some cycles on this if you want to get this in. I tried to give the PR a try by running a local ignite instance on Docker but could not send any data in it and I noticed quite a lot of exceptions on the NiFi side. Also when you want to test the thin client, you need to define a cache name, any guidance on how to set this into the Docker container?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] github-actions[bot] closed pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #4278:
URL: https://github.com/apache/nifi/pull/4278
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] matthew-formosa-gig commented on a change in pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
matthew-formosa-gig commented on a change in pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#discussion_r426157921
##########
File path: nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/test/resources/test-default-ignite-client.xml
##########
@@ -41,4 +37,12 @@
</bean>
</property>
</bean>
+
+ <bean abstract="true" id="thinIgniteClient" class="org.apache.ignite.configuration.ClientConfiguration">
+ <property name="addresses">
+ <list>
+ <value>127.0.0.1:10800</value>
Review comment:
These configurations are only being utilised in Integration Tests. These ports are the standard ports exposed by an Ignite server so it is not a matter of making the ports configurable on the client side since it really depends on the server configuration.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] github-actions[bot] closed pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #4278:
URL: https://github.com/apache/nifi/pull/4278
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] matthew-formosa-gig commented on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
matthew-formosa-gig commented on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-629654039
@joewitt Thanks for your feedback! The only dependency changes introduced are related to the Apache Ignite version bump from 1.6.0 to 2.8.0 where all transitive dependencies which changed are either related to Spring, Apache commons logging or Apache Ignite itself. As for the static ports, I have commented in the thread above.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] matthew-formosa-gig commented on a change in pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
matthew-formosa-gig commented on a change in pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#discussion_r426157921
##########
File path: nifi-nar-bundles/nifi-ignite-bundle/nifi-ignite-processors/src/test/resources/test-default-ignite-client.xml
##########
@@ -41,4 +37,12 @@
</bean>
</property>
</bean>
+
+ <bean abstract="true" id="thinIgniteClient" class="org.apache.ignite.configuration.ClientConfiguration">
+ <property name="addresses">
+ <list>
+ <value>127.0.0.1:10800</value>
Review comment:
These configurations are only being utilised in Integration Tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-922391752
We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-826170822
We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
joewitt commented on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-629648804
@matthew-formosa-gig the tests all pass and the effort put into this looks really good. I left a comment on a concern for test failures based on static ports. Otherwise my only remaining concern is whether we've reviewed all of the transitive dependency changes/updates to ensure the License and Notice files are properly updated. This is critical. have you conducted that review of the dependencies before and after this change?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] matthew-formosa-gig edited a comment on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
matthew-formosa-gig edited a comment on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-629654039
@joewitt Thanks for your feedback! The only dependency changes introduced are related to the Apache Ignite version bump from 1.6.0 to 2.8.0 where all transitive dependencies which changed are either related to Spring, Apache Commons or Apache Ignite itself. All of these were already included in the Notice files in previous commits. As for the static ports, I have commented in the thread above.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
joewitt commented on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-631647205
i didnt get to this in time and will need to come back to it in a couple weeks if nobody has. just putting this here in case someone else can pick it up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #4278: NIFI-7456: Ignite Processors - Choose between Thick and Thin Clients
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #4278:
URL: https://github.com/apache/nifi/pull/4278#issuecomment-826170822
We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org