Configuration Suggestion for NiFi HandleHTTPResponse Processor

[nairy991 <Ad...@verizon.com>]

Greetings,

I cam across an issue with the HandleHTTPResponse Processor in which the
processor sends the response but it also sends the body of the message along
with the response code. I wanted to ask if it was possible for future
development to add a configuration in the processor that allows the option
of sending the response without the body of the message, since this can
affect the time that it takes for the client to receive the response from
us, because NiFi is sending the full message again, along with the response.
As of now I have been unable to find an optimal solution that does not
impact our flow performance in regards with removing the body of the message
and only sending the response. I wanted to see if this could be a
configuration that could be looked into for this processor, since I think it
would be a useful one that could provide positive impact on the response
time of the NiFi’s HandleHTTPResponse processor.



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Configuration-Suggestion-for-NiFi-HandleHTTPResponse-Processor-tp14933.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Configuration Suggestion for NiFi HandleHTTPResponse Processor

[nairy991 <Ad...@verizon.com>]

Aldrin Piri,

Thanks for all the help, I opened the ticket in JIRA, the link for the
ticket is the following: https://issues.apache.org/jira/browse/NIFI-3529



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Configuration-Suggestion-for-NiFi-HandleHTTPResponse-Processor-tp14933p14951.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Configuration Suggestion for NiFi HandleHTTPResponse Processor

[Aldrin Piri <al...@gmail.com>]

Ah, my apologies, grabbed the wrong one!  This would be the link and/or
droid we are looking for: https://issues.apache.org/jira/browse/NIFI

Thanks!

On Fri, Feb 24, 2017 at 11:44 AM, nairy991 <Ad...@verizon.com> wrote:

> Aldrin Piri,
>
> That would be correct, we want only the attributes of the incoming flow
> file
> to be transmitted along with the associated status code, regardless of what
> status code it is. As for your second question we just want the flow to go
> to successful relationship when a 200 response is given, else all other
> responses would be consider failure in our structure. But we were thinking
> that just adding a configuration to the HandleHTTPResponse processor that
> allows to choose if the response should be sent with the content of the
> message or without it should work and be a nice improvement.
>
>
> In addition I went to the link:
> https://cwiki.apache.org/confluence/display/NIFI, could you tell me where
> I
> would need to go in this page in order to file an issue? Also would I need
> to open a JIRA ticket, is this done in this same link when an issue is
> filed?
>
>
>
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/Configuration-Suggestion-for-
> NiFi-HandleHTTPResponse-Processor-tp14933p14949.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: Configuration Suggestion for NiFi HandleHTTPResponse Processor

[nairy991 <Ad...@verizon.com>]

Aldrin Piri,

That would be correct, we want only the attributes of the incoming flow file
to be transmitted along with the associated status code, regardless of what
status code it is. As for your second question we just want the flow to go
to successful relationship when a 200 response is given, else all other
responses would be consider failure in our structure. But we were thinking
that just adding a configuration to the HandleHTTPResponse processor that
allows to choose if the response should be sent with the content of the
message or without it should work and be a nice improvement. 


In addition I went to the link:
https://cwiki.apache.org/confluence/display/NIFI, could you tell me where I
would need to go in this page in order to file an issue? Also would I need
to open a JIRA ticket, is this done in this same link when an issue is
filed?






--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Configuration-Suggestion-for-NiFi-HandleHTTPResponse-Processor-tp14933p14949.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Configuration Suggestion for NiFi HandleHTTPResponse Processor

[Aldrin Piri <al...@gmail.com>]

Thanks for the additional information.  Seems like a helpful improvement
given those conditions.

To clarify, in the issued response by HandleHTTPResponse, you would only
want the attributes of the incoming FlowFile (if any) to be transmitted and
the associated status code?

When you mention the message giving a 200 response, are you looking for the
case strictly where the message succeeds in responding, avoiding those
cases where HandleHTTPResponse would transfer a given FlowFile to failure?
Or is it the scenario where your processing establishing an HTTP Status
code after the request is received, and that is used when providing the
response through Expression Language.

Would you be able to please file an issue (
https://cwiki.apache.org/confluence/display/NIFI) to capture some of this
information and request?

--Aldrin

On Fri, Feb 24, 2017 at 10:46 AM, nairy991 <Ad...@verizon.com> wrote:

> Hi Aldrin Piri,
>
> I did try to use Replace Text processor to remove the body of the message,
> the problem is that the way the flow needs to be structure is that after
> sending a 200 positive response from the HandleHTTPResponse processor to
> the
> client, the content of the message needs to be routed to different kafka
> topics. The problem is that this routing to kafka needs to be done only if
> the message gave a 200 response, else the message needs to terminate there
> because it gave a bad response and the client needs to resend the message.
> Because of this procedure the body of the message is needed for processing
> purposes after giving the 200 response, I am just looking for a way to keep
> the content of the message but only send to the client the response without
> that content. I tried sending the content to both our HandleHTTPResponse
> processor and to a MergeContent processor and then trigger the MergeContent
> processor after the content was removed from the body of the message using
> the ReplaceText processor and then send the response, and if the response
> was successful then trigger the MergeContent processor, however this is
> still not working as well as I need it. Because of this I suggested the
> addition of a new configuration in the HandleHTTPResponse processor, in
> order to have an option that allows to choose if you want the response to
> be
> sent with the content of the message or without it, without affecting the
> content of the message in the flowfile itself so that it can be used
> directly in the rest of the flow for other purposes without the hassle of
> adding a bunch of different processors to the flow. Because the problem I
> currently have with using the MergeContent to wait for the 200 response is
> that even with only choosing the merged option and terminating the
> relationship on original and failure, it is also sending the empty content
> files of the HandleHTTPResponse processor throughout the flow along with
> the
> actual merged ones. I need a way to only send the 200 response to the
> client
> without the content but keep the content of the message to send to kafka,
> but only do this if the HandleHTTPResponse processor sends a 200 response,
> else the flow needs to terminate there.
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/Configuration-Suggestion-for-
> NiFi-HandleHTTPResponse-Processor-tp14933p14946.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: Configuration Suggestion for NiFi HandleHTTPResponse Processor

[nairy991 <Ad...@verizon.com>]

Hi Aldrin Piri,

I did try to use Replace Text processor to remove the body of the message,
the problem is that the way the flow needs to be structure is that after
sending a 200 positive response from the HandleHTTPResponse processor to the
client, the content of the message needs to be routed to different kafka
topics. The problem is that this routing to kafka needs to be done only if
the message gave a 200 response, else the message needs to terminate there
because it gave a bad response and the client needs to resend the message.
Because of this procedure the body of the message is needed for processing
purposes after giving the 200 response, I am just looking for a way to keep
the content of the message but only send to the client the response without
that content. I tried sending the content to both our HandleHTTPResponse
processor and to a MergeContent processor and then trigger the MergeContent
processor after the content was removed from the body of the message using
the ReplaceText processor and then send the response, and if the response
was successful then trigger the MergeContent processor, however this is
still not working as well as I need it. Because of this I suggested the
addition of a new configuration in the HandleHTTPResponse processor, in
order to have an option that allows to choose if you want the response to be
sent with the content of the message or without it, without affecting the
content of the message in the flowfile itself so that it can be used
directly in the rest of the flow for other purposes without the hassle of
adding a bunch of different processors to the flow. Because the problem I
currently have with using the MergeContent to wait for the 200 response is
that even with only choosing the merged option and terminating the
relationship on original and failure, it is also sending the empty content
files of the HandleHTTPResponse processor throughout the flow along with the
actual merged ones. I need a way to only send the 200 response to the client
without the content but keep the content of the message to send to kafka,
but only do this if the HandleHTTPResponse processor sends a 200 response,
else the flow needs to terminate there.



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Configuration-Suggestion-for-NiFi-HandleHTTPResponse-Processor-tp14933p14946.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Configuration Suggestion for NiFi HandleHTTPResponse Processor

[Aldrin Piri <al...@gmail.com>]

Hello!

You can get the results you are looking for by using ReplaceText to modify
the body.  In your case, you could completely replace the text with an
empty string.  There is an example of this in the Example Dataflow
Templates [1] with the Hello NiFi Web Service template.  The performance
impact in this scenario should be minimal.

Please let us know if this fulfills what you are attempting to do.  If not,
if you could provide some additional details on where this comes up short,
we can certainly discuss a better approach.

[1]
https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates

On Thu, Feb 23, 2017 at 6:15 PM, nairy991 <Ad...@verizon.com> wrote:

> Greetings,
>
> I cam across an issue with the HandleHTTPResponse Processor in which the
> processor sends the response but it also sends the body of the message
> along
> with the response code. I wanted to ask if it was possible for future
> development to add a configuration in the processor that allows the option
> of sending the response without the body of the message, since this can
> affect the time that it takes for the client to receive the response from
> us, because NiFi is sending the full message again, along with the
> response.
> As of now I have been unable to find an optimal solution that does not
> impact our flow performance in regards with removing the body of the
> message
> and only sending the response. I wanted to see if this could be a
> configuration that could be looked into for this processor, since I think
> it
> would be a useful one that could provide positive impact on the response
> time of the NiFi’s HandleHTTPResponse processor.
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/Configuration-Suggestion-for-
> NiFi-HandleHTTPResponse-Processor-tp14933.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>