You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by M Singh <ma...@yahoo.com> on 2015/11/09 01:51:41 UTC
Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Hi:
I am trying to use InvokeHTTP and have set the follow-retry to true, and have associated a self referencing relation for the InvokeHttp for no-retry (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it retires only status 500 codes requests and for 3xx does not pick up the Location header for retrying.
However, since I am new to nifi - I might be missing something or using the wrong settings or wrong processor.
If anyone has any suggestion, please let me know.
Thanks
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by Joe Percivall <jo...@yahoo.com>.
Mark,
That's fair, I think for now I'll take your approach (also easier on me) and if users want something else later I can change it.
Mans,
No problem, glad I can help.
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Monday, November 9, 2015 9:56 PM, M Singh <ma...@yahoo.com> wrote:
Joe:
No problem and thanks for your support in fixing the issue.
Mans
On Monday, November 9, 2015 12:35 PM, Joe Percivall <jo...@yahoo.com> wrote:
Mans,
I did a bit of testing using my refactor and I understand what the issue is. I was able to successfully complete the first use-case where "follow redirect" is true and within the same ontrigger follow the redirect.
The second use-case where the "follow redirect" is false didn't work as intended. It ends up routing to "No Retry" and emitting no response flowfile. The problem arises that the location header is not added to the request flowfile so there is no way to know where it was supposed to redirect to later.
This actually brings up a bigger issue that is at the core of this issue. That is when the processor reaches out to a website and routes to any of the not 2xx relationships, the headers (other than status code and message) that are returned by the response aren't stored anywhere. I'm thinking we have a property that is a regular expression that allows users to match specific headers to add as attributes to the request flowfile. Thoughts?
Not trying to discourage you from contributing but I'm going to knock this out as part of 1086. There are plenty of other tickets that could use contributors!
Thanks for bringing this up,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Monday, November 9, 2015 2:05 PM, Joe Percivall <jo...@yahoo.com> wrote:
Hey Mans,
Thanks for creating the ticket. First, have you tried the patch I supplied in 1086 [1]? I changed the underlying implementation of InvokeHttp so it very well could already be fixed.
The functionality should work where if you have "follow redirects" to true then it should follow the 3xx response to hit the other server automatically within the same ontrigger call. If "follow redirects" is false then it would not follow the redirect to the location set in the header and instead route the request to the 3xx relationship.
[1] https://issues.apache.org/jira/browse/NIFI-1086
Thanks,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Monday, November 9, 2015 1:54 PM, M Singh <ma...@yahoo.com> wrote:
Hi:
I could not find Jira ticket for InvokeHTTP processor not saving location header, so I've created a Jira ticket for this - [NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA.
My understanding is that once the http status is saved and a relationship for 3xx is available then we can route the flow file with appropriate attribute (based on location header) to the invoke http processor so it can make a new request.
Please let me know if it ok and if I can start working on it.
Thanks
[NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA
InvokeHTTP processor does not save the Location header for 3xx responses. So, we can cannot hook up the processor to route based on redirect location header values.
View on issues.apache.org Preview by Yahoo
On Monday, November 9, 2015 6:51 AM, Joe Witt <jo...@gmail.com> wrote:
Mans,
Being new to NiFi or even contributing to open source at all is not a
problem. We're here to help. Ask questions as needed and we can help
you contribute.
Thanks
Joe
On Mon, Nov 9, 2015 at 9:38 AM, M Singh <ma...@yahoo.com> wrote:
> Hi Joe:
>
> I was looking at the InvokeHttp code and I can work on enhancing the 3xx
> issue based on the pattern for handling other statuses. However, I would
> like to add a newbie caveat here.
>
> Let me know if that would help.
>
> Thanks
>
> Mans
>
>
>
> On Sunday, November 8, 2015 8:44 PM, M Singh <ma...@yahoo.com> wrote:
>
>
> Hi Joe:
>
> You are right - setting follow-redirects did not work and I did mix retry
> with redirect.
>
> I will wait for your enhancements.
>
> Thanks again for your help.
>
>
>
> On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com>
> wrote:
>
>
> Hello,
>
>
> Firstly, I think you're mixing up "retry" and "redirect". The 3xx status
> code is for redirecting to another url and 5xx is to try again. The property
> we have is "Follow Redirects". Retrying doesn't involve a location header
> but the redirect does. That being said, I did a bit of digging I don't think
> InvokeHttp was handling redirects properly. All we were doing was setting
> the "setInstanceFollowRedirects" to true, which according to this site [1]
> doesn't fully handle it.
>
> I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's
> refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test
> that out and see if that it solves the redirect and location header problem
> that would be awesome.
>
> [1]
> http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
> [2] https://issues.apache.org/jira/browse/NIFI-1086
>
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
>
> On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
>
> Joe P,
>
> Do you have any other recommendations light of the work you're doing
> now to Invoke HTTP?
>
> Thanks
> Joe
>
>
> On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
>> Thanks Joe.
>>
>> When I look at the provenance of the flow file, it shows the status as 301
>> as shown below but no Location attribute.
>>
>> If I curl the url and check the headers it does show the Location
>> attribute.
>>
>> invokehttp.status.code 301
>>
>>
>>
>>
>> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>>
>>
>> Hello
>>
>> You can use provenance or a LogAttributes processor to see what the
>> headers are of the flow file after InvokeHTTP executes. You may find
>> the location header present as one of the attributes. If so then you
>> should be able to use that attribute to establish the URL it will hit
>> next time. Does that help?
>>
>> Thanks
>> Joe
>>
>> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>>> Hi:
>>>
>>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>>> have associated a self referencing relation for the InvokeHttp for
>>> no-retry
>>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>>> retires
>>> only status 500 codes requests and for 3xx does not pick up the Location
>>> header for retrying.
>>>
>>> However, since I am new to nifi - I might be missing something or using
>>> the
>>> wrong settings or wrong processor.
>>>
>>> If anyone has any suggestion, please let me know.
>>>
>>> Thanks
>>>
>>
>>
>
>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by M Singh <ma...@yahoo.com>.
Joe:
No problem and thanks for your support in fixing the issue.
Mans
On Monday, November 9, 2015 12:35 PM, Joe Percivall <jo...@yahoo.com> wrote:
Mans,
I did a bit of testing using my refactor and I understand what the issue is. I was able to successfully complete the first use-case where "follow redirect" is true and within the same ontrigger follow the redirect.
The second use-case where the "follow redirect" is false didn't work as intended. It ends up routing to "No Retry" and emitting no response flowfile. The problem arises that the location header is not added to the request flowfile so there is no way to know where it was supposed to redirect to later.
This actually brings up a bigger issue that is at the core of this issue. That is when the processor reaches out to a website and routes to any of the not 2xx relationships, the headers (other than status code and message) that are returned by the response aren't stored anywhere. I'm thinking we have a property that is a regular expression that allows users to match specific headers to add as attributes to the request flowfile. Thoughts?
Not trying to discourage you from contributing but I'm going to knock this out as part of 1086. There are plenty of other tickets that could use contributors!
Thanks for bringing this up,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Monday, November 9, 2015 2:05 PM, Joe Percivall <jo...@yahoo.com> wrote:
Hey Mans,
Thanks for creating the ticket. First, have you tried the patch I supplied in 1086 [1]? I changed the underlying implementation of InvokeHttp so it very well could already be fixed.
The functionality should work where if you have "follow redirects" to true then it should follow the 3xx response to hit the other server automatically within the same ontrigger call. If "follow redirects" is false then it would not follow the redirect to the location set in the header and instead route the request to the 3xx relationship.
[1] https://issues.apache.org/jira/browse/NIFI-1086
Thanks,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Monday, November 9, 2015 1:54 PM, M Singh <ma...@yahoo.com> wrote:
Hi:
I could not find Jira ticket for InvokeHTTP processor not saving location header, so I've created a Jira ticket for this - [NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA.
My understanding is that once the http status is saved and a relationship for 3xx is available then we can route the flow file with appropriate attribute (based on location header) to the invoke http processor so it can make a new request.
Please let me know if it ok and if I can start working on it.
Thanks
[NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA
InvokeHTTP processor does not save the Location header for 3xx responses. So, we can cannot hook up the processor to route based on redirect location header values.
View on issues.apache.org Preview by Yahoo
On Monday, November 9, 2015 6:51 AM, Joe Witt <jo...@gmail.com> wrote:
Mans,
Being new to NiFi or even contributing to open source at all is not a
problem. We're here to help. Ask questions as needed and we can help
you contribute.
Thanks
Joe
On Mon, Nov 9, 2015 at 9:38 AM, M Singh <ma...@yahoo.com> wrote:
> Hi Joe:
>
> I was looking at the InvokeHttp code and I can work on enhancing the 3xx
> issue based on the pattern for handling other statuses. However, I would
> like to add a newbie caveat here.
>
> Let me know if that would help.
>
> Thanks
>
> Mans
>
>
>
> On Sunday, November 8, 2015 8:44 PM, M Singh <ma...@yahoo.com> wrote:
>
>
> Hi Joe:
>
> You are right - setting follow-redirects did not work and I did mix retry
> with redirect.
>
> I will wait for your enhancements.
>
> Thanks again for your help.
>
>
>
> On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com>
> wrote:
>
>
> Hello,
>
>
> Firstly, I think you're mixing up "retry" and "redirect". The 3xx status
> code is for redirecting to another url and 5xx is to try again. The property
> we have is "Follow Redirects". Retrying doesn't involve a location header
> but the redirect does. That being said, I did a bit of digging I don't think
> InvokeHttp was handling redirects properly. All we were doing was setting
> the "setInstanceFollowRedirects" to true, which according to this site [1]
> doesn't fully handle it.
>
> I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's
> refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test
> that out and see if that it solves the redirect and location header problem
> that would be awesome.
>
> [1]
> http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
> [2] https://issues.apache.org/jira/browse/NIFI-1086
>
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
>
> On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
>
> Joe P,
>
> Do you have any other recommendations light of the work you're doing
> now to Invoke HTTP?
>
> Thanks
> Joe
>
>
> On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
>> Thanks Joe.
>>
>> When I look at the provenance of the flow file, it shows the status as 301
>> as shown below but no Location attribute.
>>
>> If I curl the url and check the headers it does show the Location
>> attribute.
>>
>> invokehttp.status.code 301
>>
>>
>>
>>
>> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>>
>>
>> Hello
>>
>> You can use provenance or a LogAttributes processor to see what the
>> headers are of the flow file after InvokeHTTP executes. You may find
>> the location header present as one of the attributes. If so then you
>> should be able to use that attribute to establish the URL it will hit
>> next time. Does that help?
>>
>> Thanks
>> Joe
>>
>> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>>> Hi:
>>>
>>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>>> have associated a self referencing relation for the InvokeHttp for
>>> no-retry
>>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>>> retires
>>> only status 500 codes requests and for 3xx does not pick up the Location
>>> header for retrying.
>>>
>>> However, since I am new to nifi - I might be missing something or using
>>> the
>>> wrong settings or wrong processor.
>>>
>>> If anyone has any suggestion, please let me know.
>>>
>>> Thanks
>>>
>>
>>
>
>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by Mark Payne <ma...@hotmail.com>.
Joe,
FWIW, I would personally avoid adding a Regex property. I think we use regexes already too much. I would just add the HTTP Headers or not... or you could even add a property that specifies whether or not to add the headers as attributes.
This is very much a preference here, though. I think regexes get messy too quickly though and are very difficult to write and maintain.
-Mark
> On Nov 9, 2015, at 3:35 PM, Joe Percivall <jo...@yahoo.com> wrote:
>
> Mans,
>
> I did a bit of testing using my refactor and I understand what the issue is. I was able to successfully complete the first use-case where "follow redirect" is true and within the same ontrigger follow the redirect.
>
> The second use-case where the "follow redirect" is false didn't work as intended. It ends up routing to "No Retry" and emitting no response flowfile. The problem arises that the location header is not added to the request flowfile so there is no way to know where it was supposed to redirect to later.
>
> This actually brings up a bigger issue that is at the core of this issue. That is when the processor reaches out to a website and routes to any of the not 2xx relationships, the headers (other than status code and message) that are returned by the response aren't stored anywhere. I'm thinking we have a property that is a regular expression that allows users to match specific headers to add as attributes to the request flowfile. Thoughts?
>
> Not trying to discourage you from contributing but I'm going to knock this out as part of 1086. There are plenty of other tickets that could use contributors!
>
> Thanks for bringing this up,
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
>
> On Monday, November 9, 2015 2:05 PM, Joe Percivall <jo...@yahoo.com> wrote:
> Hey Mans,
>
> Thanks for creating the ticket. First, have you tried the patch I supplied in 1086 [1]? I changed the underlying implementation of InvokeHttp so it very well could already be fixed.
>
> The functionality should work where if you have "follow redirects" to true then it should follow the 3xx response to hit the other server automatically within the same ontrigger call. If "follow redirects" is false then it would not follow the redirect to the location set in the header and instead route the request to the 3xx relationship.
>
>
> [1] https://issues.apache.org/jira/browse/NIFI-1086
> Thanks,
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
>
>
> On Monday, November 9, 2015 1:54 PM, M Singh <ma...@yahoo.com> wrote:
>
>
>
> Hi:
>
> I could not find Jira ticket for InvokeHTTP processor not saving location header, so I've created a Jira ticket for this - [NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA.
>
> My understanding is that once the http status is saved and a relationship for 3xx is available then we can route the flow file with appropriate attribute (based on location header) to the invoke http processor so it can make a new request.
>
>
> Please let me know if it ok and if I can start working on it.
>
> Thanks
>
>
> [NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA
> InvokeHTTP processor does not save the Location header for 3xx responses. So, we can cannot hook up the processor to route based on redirect location header values.
> View on issues.apache.org Preview by Yahoo
>
>
>
>
>
> On Monday, November 9, 2015 6:51 AM, Joe Witt <jo...@gmail.com> wrote:
>
>
>
> Mans,
>
> Being new to NiFi or even contributing to open source at all is not a
> problem. We're here to help. Ask questions as needed and we can help
> you contribute.
>
> Thanks
> Joe
>
>
> On Mon, Nov 9, 2015 at 9:38 AM, M Singh <ma...@yahoo.com> wrote:
>> Hi Joe:
>>
>> I was looking at the InvokeHttp code and I can work on enhancing the 3xx
>> issue based on the pattern for handling other statuses. However, I would
>> like to add a newbie caveat here.
>>
>> Let me know if that would help.
>>
>> Thanks
>>
>> Mans
>>
>>
>>
>> On Sunday, November 8, 2015 8:44 PM, M Singh <ma...@yahoo.com> wrote:
>>
>>
>> Hi Joe:
>>
>> You are right - setting follow-redirects did not work and I did mix retry
>> with redirect.
>>
>> I will wait for your enhancements.
>>
>> Thanks again for your help.
>>
>>
>>
>> On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com>
>> wrote:
>>
>>
>> Hello,
>>
>>
>> Firstly, I think you're mixing up "retry" and "redirect". The 3xx status
>> code is for redirecting to another url and 5xx is to try again. The property
>> we have is "Follow Redirects". Retrying doesn't involve a location header
>> but the redirect does. That being said, I did a bit of digging I don't think
>> InvokeHttp was handling redirects properly. All we were doing was setting
>> the "setInstanceFollowRedirects" to true, which according to this site [1]
>> doesn't fully handle it.
>>
>> I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's
>> refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test
>> that out and see if that it solves the redirect and location header problem
>> that would be awesome.
>>
>> [1]
>> http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
>> [2] https://issues.apache.org/jira/browse/NIFI-1086
>>
>> Joe
>> - - - - - -
>> Joseph Percivall
>> linkedin.com/in/Percivall
>> e: joepercivall@yahoo.com
>>
>>
>>
>>
>> On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
>>
>>
>>
>> Joe P,
>>
>> Do you have any other recommendations light of the work you're doing
>> now to Invoke HTTP?
>>
>> Thanks
>> Joe
>>
>>
>> On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
>>> Thanks Joe.
>>>
>>> When I look at the provenance of the flow file, it shows the status as 301
>>> as shown below but no Location attribute.
>>>
>>> If I curl the url and check the headers it does show the Location
>>> attribute.
>>>
>>> invokehttp.status.code 301
>>>
>>>
>>>
>>>
>>> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>>>
>>>
>>> Hello
>>>
>>> You can use provenance or a LogAttributes processor to see what the
>>> headers are of the flow file after InvokeHTTP executes. You may find
>>> the location header present as one of the attributes. If so then you
>>> should be able to use that attribute to establish the URL it will hit
>>> next time. Does that help?
>>>
>>> Thanks
>>> Joe
>>>
>>> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>>>> Hi:
>>>>
>>>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>>>> have associated a self referencing relation for the InvokeHttp for
>>>> no-retry
>>>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>>>> retires
>>>> only status 500 codes requests and for 3xx does not pick up the Location
>>>> header for retrying.
>>>>
>>>> However, since I am new to nifi - I might be missing something or using
>>>> the
>>>> wrong settings or wrong processor.
>>>>
>>>> If anyone has any suggestion, please let me know.
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>
>>
>>
>>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by Joe Percivall <jo...@yahoo.com>.
Mans,
I did a bit of testing using my refactor and I understand what the issue is. I was able to successfully complete the first use-case where "follow redirect" is true and within the same ontrigger follow the redirect.
The second use-case where the "follow redirect" is false didn't work as intended. It ends up routing to "No Retry" and emitting no response flowfile. The problem arises that the location header is not added to the request flowfile so there is no way to know where it was supposed to redirect to later.
This actually brings up a bigger issue that is at the core of this issue. That is when the processor reaches out to a website and routes to any of the not 2xx relationships, the headers (other than status code and message) that are returned by the response aren't stored anywhere. I'm thinking we have a property that is a regular expression that allows users to match specific headers to add as attributes to the request flowfile. Thoughts?
Not trying to discourage you from contributing but I'm going to knock this out as part of 1086. There are plenty of other tickets that could use contributors!
Thanks for bringing this up,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Monday, November 9, 2015 2:05 PM, Joe Percivall <jo...@yahoo.com> wrote:
Hey Mans,
Thanks for creating the ticket. First, have you tried the patch I supplied in 1086 [1]? I changed the underlying implementation of InvokeHttp so it very well could already be fixed.
The functionality should work where if you have "follow redirects" to true then it should follow the 3xx response to hit the other server automatically within the same ontrigger call. If "follow redirects" is false then it would not follow the redirect to the location set in the header and instead route the request to the 3xx relationship.
[1] https://issues.apache.org/jira/browse/NIFI-1086
Thanks,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Monday, November 9, 2015 1:54 PM, M Singh <ma...@yahoo.com> wrote:
Hi:
I could not find Jira ticket for InvokeHTTP processor not saving location header, so I've created a Jira ticket for this - [NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA.
My understanding is that once the http status is saved and a relationship for 3xx is available then we can route the flow file with appropriate attribute (based on location header) to the invoke http processor so it can make a new request.
Please let me know if it ok and if I can start working on it.
Thanks
[NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA
InvokeHTTP processor does not save the Location header for 3xx responses. So, we can cannot hook up the processor to route based on redirect location header values.
View on issues.apache.org Preview by Yahoo
On Monday, November 9, 2015 6:51 AM, Joe Witt <jo...@gmail.com> wrote:
Mans,
Being new to NiFi or even contributing to open source at all is not a
problem. We're here to help. Ask questions as needed and we can help
you contribute.
Thanks
Joe
On Mon, Nov 9, 2015 at 9:38 AM, M Singh <ma...@yahoo.com> wrote:
> Hi Joe:
>
> I was looking at the InvokeHttp code and I can work on enhancing the 3xx
> issue based on the pattern for handling other statuses. However, I would
> like to add a newbie caveat here.
>
> Let me know if that would help.
>
> Thanks
>
> Mans
>
>
>
> On Sunday, November 8, 2015 8:44 PM, M Singh <ma...@yahoo.com> wrote:
>
>
> Hi Joe:
>
> You are right - setting follow-redirects did not work and I did mix retry
> with redirect.
>
> I will wait for your enhancements.
>
> Thanks again for your help.
>
>
>
> On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com>
> wrote:
>
>
> Hello,
>
>
> Firstly, I think you're mixing up "retry" and "redirect". The 3xx status
> code is for redirecting to another url and 5xx is to try again. The property
> we have is "Follow Redirects". Retrying doesn't involve a location header
> but the redirect does. That being said, I did a bit of digging I don't think
> InvokeHttp was handling redirects properly. All we were doing was setting
> the "setInstanceFollowRedirects" to true, which according to this site [1]
> doesn't fully handle it.
>
> I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's
> refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test
> that out and see if that it solves the redirect and location header problem
> that would be awesome.
>
> [1]
> http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
> [2] https://issues.apache.org/jira/browse/NIFI-1086
>
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
>
> On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
>
> Joe P,
>
> Do you have any other recommendations light of the work you're doing
> now to Invoke HTTP?
>
> Thanks
> Joe
>
>
> On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
>> Thanks Joe.
>>
>> When I look at the provenance of the flow file, it shows the status as 301
>> as shown below but no Location attribute.
>>
>> If I curl the url and check the headers it does show the Location
>> attribute.
>>
>> invokehttp.status.code 301
>>
>>
>>
>>
>> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>>
>>
>> Hello
>>
>> You can use provenance or a LogAttributes processor to see what the
>> headers are of the flow file after InvokeHTTP executes. You may find
>> the location header present as one of the attributes. If so then you
>> should be able to use that attribute to establish the URL it will hit
>> next time. Does that help?
>>
>> Thanks
>> Joe
>>
>> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>>> Hi:
>>>
>>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>>> have associated a self referencing relation for the InvokeHttp for
>>> no-retry
>>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>>> retires
>>> only status 500 codes requests and for 3xx does not pick up the Location
>>> header for retrying.
>>>
>>> However, since I am new to nifi - I might be missing something or using
>>> the
>>> wrong settings or wrong processor.
>>>
>>> If anyone has any suggestion, please let me know.
>>>
>>> Thanks
>>>
>>
>>
>
>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by Joe Percivall <jo...@yahoo.com>.
Hey Mans,
Thanks for creating the ticket. First, have you tried the patch I supplied in 1086 [1]? I changed the underlying implementation of InvokeHttp so it very well could already be fixed.
The functionality should work where if you have "follow redirects" to true then it should follow the 3xx response to hit the other server automatically within the same ontrigger call. If "follow redirects" is false then it would not follow the redirect to the location set in the header and instead route the request to the 3xx relationship.
[1] https://issues.apache.org/jira/browse/NIFI-1086
Thanks,
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Monday, November 9, 2015 1:54 PM, M Singh <ma...@yahoo.com> wrote:
Hi:
I could not find Jira ticket for InvokeHTTP processor not saving location header, so I've created a Jira ticket for this - [NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA.
My understanding is that once the http status is saved and a relationship for 3xx is available then we can route the flow file with appropriate attribute (based on location header) to the invoke http processor so it can make a new request.
Please let me know if it ok and if I can start working on it.
Thanks
[NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA
InvokeHTTP processor does not save the Location header for 3xx responses. So, we can cannot hook up the processor to route based on redirect location header values.
View on issues.apache.org Preview by Yahoo
On Monday, November 9, 2015 6:51 AM, Joe Witt <jo...@gmail.com> wrote:
Mans,
Being new to NiFi or even contributing to open source at all is not a
problem. We're here to help. Ask questions as needed and we can help
you contribute.
Thanks
Joe
On Mon, Nov 9, 2015 at 9:38 AM, M Singh <ma...@yahoo.com> wrote:
> Hi Joe:
>
> I was looking at the InvokeHttp code and I can work on enhancing the 3xx
> issue based on the pattern for handling other statuses. However, I would
> like to add a newbie caveat here.
>
> Let me know if that would help.
>
> Thanks
>
> Mans
>
>
>
> On Sunday, November 8, 2015 8:44 PM, M Singh <ma...@yahoo.com> wrote:
>
>
> Hi Joe:
>
> You are right - setting follow-redirects did not work and I did mix retry
> with redirect.
>
> I will wait for your enhancements.
>
> Thanks again for your help.
>
>
>
> On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com>
> wrote:
>
>
> Hello,
>
>
> Firstly, I think you're mixing up "retry" and "redirect". The 3xx status
> code is for redirecting to another url and 5xx is to try again. The property
> we have is "Follow Redirects". Retrying doesn't involve a location header
> but the redirect does. That being said, I did a bit of digging I don't think
> InvokeHttp was handling redirects properly. All we were doing was setting
> the "setInstanceFollowRedirects" to true, which according to this site [1]
> doesn't fully handle it.
>
> I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's
> refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test
> that out and see if that it solves the redirect and location header problem
> that would be awesome.
>
> [1]
> http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
> [2] https://issues.apache.org/jira/browse/NIFI-1086
>
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
>
> On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
>
> Joe P,
>
> Do you have any other recommendations light of the work you're doing
> now to Invoke HTTP?
>
> Thanks
> Joe
>
>
> On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
>> Thanks Joe.
>>
>> When I look at the provenance of the flow file, it shows the status as 301
>> as shown below but no Location attribute.
>>
>> If I curl the url and check the headers it does show the Location
>> attribute.
>>
>> invokehttp.status.code 301
>>
>>
>>
>>
>> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>>
>>
>> Hello
>>
>> You can use provenance or a LogAttributes processor to see what the
>> headers are of the flow file after InvokeHTTP executes. You may find
>> the location header present as one of the attributes. If so then you
>> should be able to use that attribute to establish the URL it will hit
>> next time. Does that help?
>>
>> Thanks
>> Joe
>>
>> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>>> Hi:
>>>
>>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>>> have associated a self referencing relation for the InvokeHttp for
>>> no-retry
>>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>>> retires
>>> only status 500 codes requests and for 3xx does not pick up the Location
>>> header for retrying.
>>>
>>> However, since I am new to nifi - I might be missing something or using
>>> the
>>> wrong settings or wrong processor.
>>>
>>> If anyone has any suggestion, please let me know.
>>>
>>> Thanks
>>>
>>
>>
>
>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by M Singh <ma...@yahoo.com>.
Hi:
I could not find Jira ticket for InvokeHTTP processor not saving location header, so I've created a Jira ticket for this - [NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRA.
My understanding is that once the http status is saved and a relationship for 3xx is available then we can route the flow file with appropriate attribute (based on location header) to the invoke http processor so it can make a new request.
Please let me know if it ok and if I can start working on it.
Thanks
| |
| | | | | |
| [NIFI-1133] InvokeHTTP Processor does not save Location header for 3xx responses - ASF JIRAInvokeHTTP processor does not save the Location header for 3xx responses. So, we can cannot hook up the processor to route based on redirect location header values. |
| |
| View on issues.apache.org | Preview by Yahoo |
| |
| |
On Monday, November 9, 2015 6:51 AM, Joe Witt <jo...@gmail.com> wrote:
Mans,
Being new to NiFi or even contributing to open source at all is not a
problem. We're here to help. Ask questions as needed and we can help
you contribute.
Thanks
Joe
On Mon, Nov 9, 2015 at 9:38 AM, M Singh <ma...@yahoo.com> wrote:
> Hi Joe:
>
> I was looking at the InvokeHttp code and I can work on enhancing the 3xx
> issue based on the pattern for handling other statuses. However, I would
> like to add a newbie caveat here.
>
> Let me know if that would help.
>
> Thanks
>
> Mans
>
>
>
> On Sunday, November 8, 2015 8:44 PM, M Singh <ma...@yahoo.com> wrote:
>
>
> Hi Joe:
>
> You are right - setting follow-redirects did not work and I did mix retry
> with redirect.
>
> I will wait for your enhancements.
>
> Thanks again for your help.
>
>
>
> On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com>
> wrote:
>
>
> Hello,
>
>
> Firstly, I think you're mixing up "retry" and "redirect". The 3xx status
> code is for redirecting to another url and 5xx is to try again. The property
> we have is "Follow Redirects". Retrying doesn't involve a location header
> but the redirect does. That being said, I did a bit of digging I don't think
> InvokeHttp was handling redirects properly. All we were doing was setting
> the "setInstanceFollowRedirects" to true, which according to this site [1]
> doesn't fully handle it.
>
> I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's
> refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test
> that out and see if that it solves the redirect and location header problem
> that would be awesome.
>
> [1]
> http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
> [2] https://issues.apache.org/jira/browse/NIFI-1086
>
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
>
> On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
>
> Joe P,
>
> Do you have any other recommendations light of the work you're doing
> now to Invoke HTTP?
>
> Thanks
> Joe
>
>
> On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
>> Thanks Joe.
>>
>> When I look at the provenance of the flow file, it shows the status as 301
>> as shown below but no Location attribute.
>>
>> If I curl the url and check the headers it does show the Location
>> attribute.
>>
>> invokehttp.status.code 301
>>
>>
>>
>>
>> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>>
>>
>> Hello
>>
>> You can use provenance or a LogAttributes processor to see what the
>> headers are of the flow file after InvokeHTTP executes. You may find
>> the location header present as one of the attributes. If so then you
>> should be able to use that attribute to establish the URL it will hit
>> next time. Does that help?
>>
>> Thanks
>> Joe
>>
>> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>>> Hi:
>>>
>>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>>> have associated a self referencing relation for the InvokeHttp for
>>> no-retry
>>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>>> retires
>>> only status 500 codes requests and for 3xx does not pick up the Location
>>> header for retrying.
>>>
>>> However, since I am new to nifi - I might be missing something or using
>>> the
>>> wrong settings or wrong processor.
>>>
>>> If anyone has any suggestion, please let me know.
>>>
>>> Thanks
>>>
>>
>>
>
>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by Joe Witt <jo...@gmail.com>.
Mans,
Being new to NiFi or even contributing to open source at all is not a
problem. We're here to help. Ask questions as needed and we can help
you contribute.
Thanks
Joe
On Mon, Nov 9, 2015 at 9:38 AM, M Singh <ma...@yahoo.com> wrote:
> Hi Joe:
>
> I was looking at the InvokeHttp code and I can work on enhancing the 3xx
> issue based on the pattern for handling other statuses. However, I would
> like to add a newbie caveat here.
>
> Let me know if that would help.
>
> Thanks
>
> Mans
>
>
>
> On Sunday, November 8, 2015 8:44 PM, M Singh <ma...@yahoo.com> wrote:
>
>
> Hi Joe:
>
> You are right - setting follow-redirects did not work and I did mix retry
> with redirect.
>
> I will wait for your enhancements.
>
> Thanks again for your help.
>
>
>
> On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com>
> wrote:
>
>
> Hello,
>
>
> Firstly, I think you're mixing up "retry" and "redirect". The 3xx status
> code is for redirecting to another url and 5xx is to try again. The property
> we have is "Follow Redirects". Retrying doesn't involve a location header
> but the redirect does. That being said, I did a bit of digging I don't think
> InvokeHttp was handling redirects properly. All we were doing was setting
> the "setInstanceFollowRedirects" to true, which according to this site [1]
> doesn't fully handle it.
>
> I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's
> refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test
> that out and see if that it solves the redirect and location header problem
> that would be awesome.
>
> [1]
> http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
> [2] https://issues.apache.org/jira/browse/NIFI-1086
>
> Joe
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
>
> On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
>
> Joe P,
>
> Do you have any other recommendations light of the work you're doing
> now to Invoke HTTP?
>
> Thanks
> Joe
>
>
> On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
>> Thanks Joe.
>>
>> When I look at the provenance of the flow file, it shows the status as 301
>> as shown below but no Location attribute.
>>
>> If I curl the url and check the headers it does show the Location
>> attribute.
>>
>> invokehttp.status.code 301
>>
>>
>>
>>
>> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>>
>>
>> Hello
>>
>> You can use provenance or a LogAttributes processor to see what the
>> headers are of the flow file after InvokeHTTP executes. You may find
>> the location header present as one of the attributes. If so then you
>> should be able to use that attribute to establish the URL it will hit
>> next time. Does that help?
>>
>> Thanks
>> Joe
>>
>> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>>> Hi:
>>>
>>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>>> have associated a self referencing relation for the InvokeHttp for
>>> no-retry
>>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>>> retires
>>> only status 500 codes requests and for 3xx does not pick up the Location
>>> header for retrying.
>>>
>>> However, since I am new to nifi - I might be missing something or using
>>> the
>>> wrong settings or wrong processor.
>>>
>>> If anyone has any suggestion, please let me know.
>>>
>>> Thanks
>>>
>>
>>
>
>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by M Singh <ma...@yahoo.com>.
Hi Joe:
I was looking at the InvokeHttp code and I can work on enhancing the 3xx issue based on the pattern for handling other statuses. However, I would like to add a newbie caveat here.
Let me know if that would help.
Thanks
Mans
On Sunday, November 8, 2015 8:44 PM, M Singh <ma...@yahoo.com> wrote:
Hi Joe:
You are right - setting follow-redirects did not work and I did mix retry with redirect.
I will wait for your enhancements.
Thanks again for your help.
On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com> wrote:
Hello,
Firstly, I think you're mixing up "retry" and "redirect". The 3xx status code is for redirecting to another url and 5xx is to try again. The property we have is "Follow Redirects". Retrying doesn't involve a location header but the redirect does. That being said, I did a bit of digging I don't think InvokeHttp was handling redirects properly. All we were doing was setting the "setInstanceFollowRedirects" to true, which according to this site [1] doesn't fully handle it.
I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test that out and see if that it solves the redirect and location header problem that would be awesome.
[1] http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
[2] https://issues.apache.org/jira/browse/NIFI-1086
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
Joe P,
Do you have any other recommendations light of the work you're doing
now to Invoke HTTP?
Thanks
Joe
On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
> Thanks Joe.
>
> When I look at the provenance of the flow file, it shows the status as 301
> as shown below but no Location attribute.
>
> If I curl the url and check the headers it does show the Location attribute.
>
> invokehttp.status.code 301
>
>
>
>
> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
> Hello
>
> You can use provenance or a LogAttributes processor to see what the
> headers are of the flow file after InvokeHTTP executes. You may find
> the location header present as one of the attributes. If so then you
> should be able to use that attribute to establish the URL it will hit
> next time. Does that help?
>
> Thanks
> Joe
>
> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>> Hi:
>>
>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>> have associated a self referencing relation for the InvokeHttp for
>> no-retry
>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>> retires
>> only status 500 codes requests and for 3xx does not pick up the Location
>> header for retrying.
>>
>> However, since I am new to nifi - I might be missing something or using
>> the
>> wrong settings or wrong processor.
>>
>> If anyone has any suggestion, please let me know.
>>
>> Thanks
>>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by M Singh <ma...@yahoo.com>.
Hi Joe:
You are right - setting follow-redirects did not work and I did mix retry with redirect.
I will wait for your enhancements.
Thanks again for your help.
On Sunday, November 8, 2015 8:31 PM, Joe Percivall <jo...@yahoo.com> wrote:
Hello,
Firstly, I think you're mixing up "retry" and "redirect". The 3xx status code is for redirecting to another url and 5xx is to try again. The property we have is "Follow Redirects". Retrying doesn't involve a location header but the redirect does. That being said, I did a bit of digging I don't think InvokeHttp was handling redirects properly. All we were doing was setting the "setInstanceFollowRedirects" to true, which according to this site [1] doesn't fully handle it.
I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test that out and see if that it solves the redirect and location header problem that would be awesome.
[1] http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
[2] https://issues.apache.org/jira/browse/NIFI-1086
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
Joe P,
Do you have any other recommendations light of the work you're doing
now to Invoke HTTP?
Thanks
Joe
On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
> Thanks Joe.
>
> When I look at the provenance of the flow file, it shows the status as 301
> as shown below but no Location attribute.
>
> If I curl the url and check the headers it does show the Location attribute.
>
> invokehttp.status.code 301
>
>
>
>
> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
> Hello
>
> You can use provenance or a LogAttributes processor to see what the
> headers are of the flow file after InvokeHTTP executes. You may find
> the location header present as one of the attributes. If so then you
> should be able to use that attribute to establish the URL it will hit
> next time. Does that help?
>
> Thanks
> Joe
>
> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>> Hi:
>>
>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>> have associated a self referencing relation for the InvokeHttp for
>> no-retry
>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>> retires
>> only status 500 codes requests and for 3xx does not pick up the Location
>> header for retrying.
>>
>> However, since I am new to nifi - I might be missing something or using
>> the
>> wrong settings or wrong processor.
>>
>> If anyone has any suggestion, please let me know.
>>
>> Thanks
>>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by Joe Percivall <jo...@yahoo.com>.
Hello,
Firstly, I think you're mixing up "retry" and "redirect". The 3xx status code is for redirecting to another url and 5xx is to try again. The property we have is "Follow Redirects". Retrying doesn't involve a location header but the redirect does. That being said, I did a bit of digging I don't think InvokeHttp was handling redirects properly. All we were doing was setting the "setInstanceFollowRedirects" to true, which according to this site [1] doesn't fully handle it.
I am going to attach a patch to ticket 1086[2] tonight for InvokeHttp's refactor to use OkHttp instead of HttpUrlConnection. If you'd like to test that out and see if that it solves the redirect and location header problem that would be awesome.
[1] http://www.mkyong.com/java/java-httpurlconnection-follow-redirect-example/
[2] https://issues.apache.org/jira/browse/NIFI-1086
Joe
- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joepercivall@yahoo.com
On Sunday, November 8, 2015 9:53 PM, Joe Witt <jo...@gmail.com> wrote:
Joe P,
Do you have any other recommendations light of the work you're doing
now to Invoke HTTP?
Thanks
Joe
On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
> Thanks Joe.
>
> When I look at the provenance of the flow file, it shows the status as 301
> as shown below but no Location attribute.
>
> If I curl the url and check the headers it does show the Location attribute.
>
> invokehttp.status.code 301
>
>
>
>
> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
> Hello
>
> You can use provenance or a LogAttributes processor to see what the
> headers are of the flow file after InvokeHTTP executes. You may find
> the location header present as one of the attributes. If so then you
> should be able to use that attribute to establish the URL it will hit
> next time. Does that help?
>
> Thanks
> Joe
>
> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>> Hi:
>>
>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>> have associated a self referencing relation for the InvokeHttp for
>> no-retry
>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>> retires
>> only status 500 codes requests and for 3xx does not pick up the Location
>> header for retrying.
>>
>> However, since I am new to nifi - I might be missing something or using
>> the
>> wrong settings or wrong processor.
>>
>> If anyone has any suggestion, please let me know.
>>
>> Thanks
>>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by Joe Witt <jo...@gmail.com>.
Joe P,
Do you have any other recommendations light of the work you're doing
now to Invoke HTTP?
Thanks
Joe
On Sun, Nov 8, 2015 at 9:49 PM, M Singh <ma...@yahoo.com> wrote:
> Thanks Joe.
>
> When I look at the provenance of the flow file, it shows the status as 301
> as shown below but no Location attribute.
>
> If I curl the url and check the headers it does show the Location attribute.
>
> invokehttp.status.code 301
>
>
>
>
> On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
>
>
> Hello
>
> You can use provenance or a LogAttributes processor to see what the
> headers are of the flow file after InvokeHTTP executes. You may find
> the location header present as one of the attributes. If so then you
> should be able to use that attribute to establish the URL it will hit
> next time. Does that help?
>
> Thanks
> Joe
>
> On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
>> Hi:
>>
>> I am trying to use InvokeHTTP and have set the follow-retry to true, and
>> have associated a self referencing relation for the InvokeHttp for
>> no-retry
>> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it
>> retires
>> only status 500 codes requests and for 3xx does not pick up the Location
>> header for retrying.
>>
>> However, since I am new to nifi - I might be missing something or using
>> the
>> wrong settings or wrong processor.
>>
>> If anyone has any suggestion, please let me know.
>>
>> Thanks
>>
>
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by M Singh <ma...@yahoo.com>.
Thanks Joe.
When I look at the provenance of the flow file, it shows the status as 301 as shown below but no Location attribute.
If I curl the url and check the headers it does show the Location attribute.
invokehttp.status.code 301
On Sunday, November 8, 2015 4:58 PM, Joe Witt <jo...@gmail.com> wrote:
Hello
You can use provenance or a LogAttributes processor to see what the
headers are of the flow file after InvokeHTTP executes. You may find
the location header present as one of the attributes. If so then you
should be able to use that attribute to establish the URL it will hit
next time. Does that help?
Thanks
Joe
On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
> Hi:
>
> I am trying to use InvokeHTTP and have set the follow-retry to true, and
> have associated a self referencing relation for the InvokeHttp for no-retry
> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it retires
> only status 500 codes requests and for 3xx does not pick up the Location
> header for retrying.
>
> However, since I am new to nifi - I might be missing something or using the
> wrong settings or wrong processor.
>
> If anyone has any suggestion, please let me know.
>
> Thanks
>
Re: Nifi - InvokeHTTP - retrying on 3xx with Location header issue
Posted by Joe Witt <jo...@gmail.com>.
Hello
You can use provenance or a LogAttributes processor to see what the
headers are of the flow file after InvokeHTTP executes. You may find
the location header present as one of the attributes. If so then you
should be able to use that attribute to establish the URL it will hit
next time. Does that help?
Thanks
Joe
On Sun, Nov 8, 2015 at 7:51 PM, M Singh <ma...@yahoo.com> wrote:
> Hi:
>
> I am trying to use InvokeHTTP and have set the follow-retry to true, and
> have associated a self referencing relation for the InvokeHttp for no-retry
> (1xx, 2xx, and 3xx) and retry (5xx) relations. But it looks like it retires
> only status 500 codes requests and for 3xx does not pick up the Location
> header for retrying.
>
> However, since I am new to nifi - I might be missing something or using the
> wrong settings or wrong processor.
>
> If anyone has any suggestion, please let me know.
>
> Thanks
>