You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2020/01/09 14:33:35 UTC
[GitHub] [nifi] dirkbig opened a new pull request #3971: NIFI-6997:
consumeAMQP closing connection when queue not found
dirkbig opened a new pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971
code changes fixes connection bug wherein
- consumeAMQP processor keeps opening tcp-connections while trying to connect to a queue that cannot be found/does not exist.
- every try results in more connections that are not being closed (immediately)
added code change makes use of the provided connection.close() function inorder to close the connection once connection to queue failed.
see: https://issues.apache.org/jira/projects/NIFI/issues/NIFI-6997
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
#### Description of PR
_Enables X functionality; fixes bug NIFI-YYYY._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] joewitt commented on issue #3971: NIFI-6997: consumeAMQP
closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
joewitt commented on issue #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#issuecomment-573043464
no checklist needed.
exceptions should not be at info level.
good to use common naming conventions of camel case rather than hyphen
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dirkbig removed a comment on issue #3971: NIFI-6997:
consumeAMQP closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
dirkbig removed a comment on issue #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#issuecomment-574096997
> Although the suggested modification solves the issue of connections piling up, there are still a lot of requests to the server (though they will be closed immediately). Wouldn't it be better if once the processor fails to connect, it tries again only after a certain period of time and not immediately? For example, the processor could try again after 100ms, if it still fails after 200ms, then after 400ms and so on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ravowlga123 commented on a change in pull request #3971:
NIFI-6997: consumeAMQP closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
ravowlga123 commented on a change in pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#discussion_r365238434
##########
File path: nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/ConsumeAMQP.java
##########
@@ -194,6 +194,13 @@ protected synchronized AMQPConsumer createAMQPWorker(final ProcessContext contex
return amqpConsumer;
} catch (final IOException ioe) {
+ try {
+ connection.close();
+ getLogger().warn("Closed connection at port " + connection.getPort());
+ } catch (final IOException ioe_close) {
Review comment:
Instead of "ioe_close" "ioeClose" would be better?? Also not sure if log message should be at warn or info level.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dirkbig opened a new pull request #3971: NIFI-6997:
consumeAMQP closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
dirkbig opened a new pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971
code changes fixes connection bug wherein
- consumeAMQP processor keeps opening tcp-connections while trying to connect to a queue that cannot be found/does not exist.
- every try results in more connections that are not being closed (immediately)
added code change makes use of the provided connection.close() function inorder to close the connection once connection to queue failed.
see: https://issues.apache.org/jira/projects/NIFI/issues/NIFI-6997
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dirkbig commented on issue #3971: NIFI-6997: consumeAMQP
closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
dirkbig commented on issue #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#issuecomment-574096997
> Although the suggested modification solves the issue of connections piling up, there are still a lot of requests to the server (though they will be closed immediately). Wouldn't it be better if once the processor fails to connect, it tries again only after a certain period of time and not immediately? For example, the processor could try again after 100ms, if it still fails after 200ms, then after 400ms and so on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dirkbig commented on a change in pull request #3971:
NIFI-6997: consumeAMQP closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
dirkbig commented on a change in pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#discussion_r366239421
##########
File path: nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/ConsumeAMQP.java
##########
@@ -194,6 +194,13 @@ protected synchronized AMQPConsumer createAMQPWorker(final ProcessContext contex
return amqpConsumer;
} catch (final IOException ioe) {
+ try {
+ connection.close();
+ getLogger().warn("Closed connection at port " + connection.getPort());
+ } catch (final IOException ioeClose) {
+ throw new ProcessException("Failed to close connection at port " + connection.getPort());
Review comment:
would it be a good idea to swallow the ioeClose exception by means of a warn-log? (and indeed remove the previous logging of the succesful closing of connection:
```java
} catch (final IOException ioe) {
try {
connection.close();
} catch (final IOException ioeClose) {
getLogger().warn("Failed to close connection at port " + connection.getPort(), ioeClose);
}
throw new ProcessException("Failed to connect to AMQP Broker", ioe);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] joewitt commented on a change in pull request #3971:
NIFI-6997: consumeAMQP closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
joewitt commented on a change in pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#discussion_r365281042
##########
File path: nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/ConsumeAMQP.java
##########
@@ -194,6 +194,13 @@ protected synchronized AMQPConsumer createAMQPWorker(final ProcessContext contex
return amqpConsumer;
} catch (final IOException ioe) {
+ try {
+ connection.close();
+ getLogger().warn("Closed connection at port " + connection.getPort());
+ } catch (final IOException ioeClose) {
+ throw new ProcessException("Failed to close connection at port " + connection.getPort());
Review comment:
You're at this point because of an io exception that already occurred. Here you're closing a connection as good practice but the fact that it too might not go well isn't useful/not something you could do anything about. As such I'd closeQuietly meaning do not rethrow the exception and do not log even at a warn level. If you want you could keep the logging part and only as a debug thing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] asfgit closed pull request #3971: NIFI-6997: consumeAMQP
closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dirkbig closed pull request #3971: NIFI-6997: consumeAMQP
closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
dirkbig closed pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dirkbig commented on a change in pull request #3971:
NIFI-6997: consumeAMQP closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
dirkbig commented on a change in pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#discussion_r366239421
##########
File path: nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/ConsumeAMQP.java
##########
@@ -194,6 +194,13 @@ protected synchronized AMQPConsumer createAMQPWorker(final ProcessContext contex
return amqpConsumer;
} catch (final IOException ioe) {
+ try {
+ connection.close();
+ getLogger().warn("Closed connection at port " + connection.getPort());
+ } catch (final IOException ioeClose) {
+ throw new ProcessException("Failed to close connection at port " + connection.getPort());
Review comment:
would it be a good idea to swallow the ioeClose exception by means of a warn-log?
```java
} catch (final IOException ioe) {
try {
connection.close();
} catch (final IOException ioeClose) {
getLogger().warn("Failed to close connection at port " + connection.getPort(), ioeClose);
}
throw new ProcessException("Failed to connect to AMQP Broker", ioe);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] ravowlga123 commented on issue #3971: NIFI-6997: consumeAMQP
closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
ravowlga123 commented on issue #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#issuecomment-573038910
Hey @dirkbig I don't see the checklist which is usually present while opening the PR regarding the changes made. Would you please add it??
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] dirkbig commented on a change in pull request #3971:
NIFI-6997: consumeAMQP closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
dirkbig commented on a change in pull request #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#discussion_r365272278
##########
File path: nifi-nar-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/ConsumeAMQP.java
##########
@@ -194,6 +194,13 @@ protected synchronized AMQPConsumer createAMQPWorker(final ProcessContext contex
return amqpConsumer;
} catch (final IOException ioe) {
+ try {
+ connection.close();
+ getLogger().warn("Closed connection at port " + connection.getPort());
+ } catch (final IOException ioe_close) {
Review comment:
thanks for the input, just changed it. Kept the log message to level WARN as joewitt affirmed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] mohofe commented on issue #3971: NIFI-6997: consumeAMQP
closing connection when queue not found
Posted by GitBox <gi...@apache.org>.
mohofe commented on issue #3971: NIFI-6997: consumeAMQP closing connection when queue not found
URL: https://github.com/apache/nifi/pull/3971#issuecomment-574089758
Although the suggested modification solves the issue of connections piling up, there are still a lot of requests to the server (though they will be closed immediately). Wouldn't it be better if once the processor fails to connect, it tries again only after a certain period of time and not immediately? For example, the processor could try again after 100ms, if it still fails after 200ms, then after 400ms and so on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services