You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by bbende <gi...@git.apache.org> on 2016/01/29 17:20:40 UTC
[GitHub] nifi pull request: NIFI-1417 Exposing several connection settings ...
GitHub user bbende opened a pull request:
https://github.com/apache/nifi/pull/194
NIFI-1417 Exposing several connection settings on the Solr processors…
I was not able to reproduce the exact scenario described in this ticket, but based on the stack trace I believe exposing some of the connection setting on the Solr processors would resolve this issue.
The properties exposed are:
Solr Socket Timeout = The socket timeout on the underlying HttpClient
Solr Connection Timeout = The connection timeout on the underlying HttpClient
Solr Maximum Connections = The max connections on the underlying HttpClient
Solr Maximum Connections Per Host = The max connections per host on the underlying HttpClient
ZooKeeper Timeout = The timeout on the ZK client
ZooKeeper Connection Timeout = The timeout on the ZK client connecting
It seemed kinda of redundant to prefix some of them with "Solr ..." but I wanted to be clear which connection the property was for (ZK vs Solr).
The Max Connections and Max Connections Per Host were not totally necessary, however SolrJ was specifically setting these when using standard Solr and not passing in an HttpClient. So now that we are passing in a custom HttpClient, I was worried about not providing a way to control these values.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bbende/nifi NIFI-1417
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/194.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #194
----
commit eda3540b657d64b0e8cdd98c9c2f7103df6da453
Author: Bryan Bende <bb...@apache.org>
Date: 2016-01-29T16:12:30Z
NIFI-1417 Exposing several connection settings on the Solr processors to help deal with timeouts and adding an @OnStopped method to close the SolrClient which didn't appear to be happening before
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1417 Exposing several connection settings ...
Posted by bbende <gi...@git.apache.org>.
Github user bbende commented on the pull request:
https://github.com/apache/nifi/pull/194#issuecomment-178209934
@markap14 thanks for reviewing!
Good point about the isSet(). I was thinking someone could remove the default value, but didn't realize that isn't possible. I'll clean that up and commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1417 Exposing several connection settings ...
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/194#discussion_r51478207
--- Diff: nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/SolrProcessor.java ---
@@ -120,6 +202,22 @@ protected final SolrClient getSolrClient() {
.explanation("A collection must specified for Solr Type of Cloud")
.build());
}
+
+ if (!context.getProperty(ZK_CLIENT_TIMEOUT).isSet()) {
--- End diff --
This is 'dead code' as isSet() will always return 'true', since there is a defaultValue
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1417 Exposing several connection settings ...
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/194#discussion_r51478213
--- Diff: nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/SolrProcessor.java ---
@@ -120,6 +202,22 @@ protected final SolrClient getSolrClient() {
.explanation("A collection must specified for Solr Type of Cloud")
.build());
}
+
+ if (!context.getProperty(ZK_CLIENT_TIMEOUT).isSet()) {
+ problems.add(new ValidationResult.Builder()
+ .subject(ZK_CLIENT_TIMEOUT.getName())
+ .valid(false)
+ .explanation("A ZooKeeper Client Timeout must specified for Solr Type of Cloud")
+ .build());
+ }
+
+ if (!context.getProperty(ZK_CONNECTION_TIMEOUT).isSet()) {
--- End diff --
This is 'dead code' as isSet() will always return 'true', since there is a defaultValue
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1417 Exposing several connection settings ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/194
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1417 Exposing several connection settings ...
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on the pull request:
https://github.com/apache/nifi/pull/194#issuecomment-178187346
I don't know how to really reproduce the issue either, but i agree that these are good properties to introduce. All new properties have defaults so flows will not be broken. Unit tests pass and contrib-check passes. While I don't think the added checks are necessary in the customValidate, all else looks good. +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---