You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by mgaido91 <gi...@git.apache.org> on 2018/03/14 15:30:43 UTC
[GitHub] nifi pull request #2544: NIFI-4959: Remove flowfiles and close connection fo...
GitHub user mgaido91 opened a pull request:
https://github.com/apache/nifi/pull/2544
NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes? Tested manually
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? NA
- [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? NA
- [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? NA
- [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? NA
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/nifi NIFI-4959
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2544.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2544
----
commit 15f0a04a1b2b4600ccf2b43b83f0ec157dbfd500
Author: Marco Gaido <ma...@...>
Date: 2018-03-14T15:27:17Z
NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException
----
---
[GitHub] nifi pull request #2544: NIFI-4959: Remove flowfiles and close connection fo...
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2544#discussion_r174922021
--- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java ---
@@ -520,6 +521,27 @@ public void onTrigger(final ProcessContext context, final ProcessSession session
new Object[]{request.getRemoteAddr(), e});
session.remove(flowFile);
return;
+ } catch (final FlowFileAccessException e) {
+ // some bad requests can produce a IOException on the HTTP stream, which makes a FlowFileAccessException to
+ // be thrown. We should handle these cases here, while other FlowFileAccessException are re-thrown
+ if (!(e.getCause() != null && e.getCause() instanceof FlowFileAccessException
--- End diff --
This logic seems very specific to me. I'm afraid that it's also quite brittle, as well, because the wrapping of those exceptions could change at any time. I *think* the idea here is "If you hit an IOException when reading from the HTTP Request, then send back a BAD REQUEST status code. Else, rethrow the Exception." Correct?
If so, we could make this a little cleaner and more stable, IMO, if we change the logic above a little bit. Instead of calling `ProcessSession.importFrom(InputStream, FlowFile)` (which would throw FlowFileAccessException if any IOException is thrown), we can change it to something like:
```
try (OutputStream flowFileOut = session.write(flowFile)) {
StreamUtils.copy(request.getInputStream(), flowFileOut);
} catch (IOException ioe) {
// this will occur only if an IOException is thrown from reading the InputStream. If an IOException is thrown from
// the OutputStream, the OutputStream will always wrap it with a FlowFileAccessException because this is
// a "special case" where we are writing to the Content Repo.
// ... new logic here to remove flowfile and send back bad request status code
}
```
---
[GitHub] nifi issue #2544: NIFI-4959: Remove flowfiles and close connection for Bad R...
Posted by mgaido91 <gi...@git.apache.org>.
Github user mgaido91 commented on the issue:
https://github.com/apache/nifi/pull/2544
Thanks for your help @markap14
---
[GitHub] nifi pull request #2544: NIFI-4959: Remove flowfiles and close connection fo...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2544
---
[GitHub] nifi pull request #2544: NIFI-4959: Remove flowfiles and close connection fo...
Posted by mgaido91 <gi...@git.apache.org>.
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2544#discussion_r175079382
--- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java ---
@@ -520,6 +521,27 @@ public void onTrigger(final ProcessContext context, final ProcessSession session
new Object[]{request.getRemoteAddr(), e});
session.remove(flowFile);
return;
+ } catch (final FlowFileAccessException e) {
+ // some bad requests can produce a IOException on the HTTP stream, which makes a FlowFileAccessException to
+ // be thrown. We should handle these cases here, while other FlowFileAccessException are re-thrown
+ if (!(e.getCause() != null && e.getCause() instanceof FlowFileAccessException
--- End diff --
yes, thank you very much for your comment and your help @markap14 . I am updating the PR accordingly. Thanks.
---
[GitHub] nifi issue #2544: NIFI-4959: Remove flowfiles and close connection for Bad R...
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/2544
@mgaido91 i agree, i addressed the JMS tests on master. Will take a look at this now. Thanks!~
---
[GitHub] nifi issue #2544: NIFI-4959: Remove flowfiles and close connection for Bad R...
Posted by mgaido91 <gi...@git.apache.org>.
Github user mgaido91 commented on the issue:
https://github.com/apache/nifi/pull/2544
the test error is unrelated:
```
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ nifi-hive-processors ---
[ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 13.251 s <<< FAILURE! - in org.apache.nifi.jms.processors.PublishJMSTest
[ERROR] validateSuccessfulPublishAndTransferToSuccessWithEL(org.apache.nifi.jms.processors.PublishJMSTest) Time elapsed: 10.007 s <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 10000 milliseconds
at org.apache.nifi.jms.processors.PublishJMSTest.validateSuccessfulPublishAndTransferToSuccessWithEL(PublishJMSTest.java:103)
```
---
[GitHub] nifi issue #2544: NIFI-4959: Remove flowfiles and close connection for Bad R...
Posted by markap14 <gi...@git.apache.org>.
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/2544
@mgaido91 all looks good to me. Thanks for addressing this! I've merged it to master.
---