You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Etienne Jouvin <la...@gmail.com> on 2020/02/10 13:47:53 UTC
On ExecuteSQL (1.9.2) failure, infinite loop
Hello All.
Here is an extract of my process
[image: image.png]
If the executeSQL failed (invalid SQL for example), the flowfile goes to
the failure relation.
At very first, I call the update attribute and I saw that the flowfile is
kept into the relation, never proceeded.
So I try to put intermediate processor, LogMessage, but this is the same.
Notice that I do not have this in the success relation.
Does someone have this also ?
Regards
Etienne Jouvin
Re: On ExecuteSQL (1.9.2) failure, infinite loop
Posted by Etienne Jouvin <la...@gmail.com>.
Ok.
That's not m'y usecase, si put penalty to 0.
Understood, thanks a lot
Le lun. 10 févr. 2020 à 17:29, Pierre Villard <pi...@gmail.com>
a écrit :
> Etienne,
>
> The penalty duration is particularly useful when you have the relationship
> going back to the ExecuteSQL processor (self-loop). In that case, you don't
> want to constantly hit the database and give some time before trying again.
>
> HTH,
> Pierre
>
> Le lun. 10 févr. 2020 à 06:03, Etienne Jouvin <la...@gmail.com> a
> écrit :
>
>> Put 0 seconds to penalized and it goes to the logger processor without
>> waiting.
>>
>> That's fine.
>>
>> I just do not understand weel why there is a penalty or yield failure,
>> but now that I know this, this is ok.
>>
>>
>> Le lun. 10 févr. 2020 à 14:58, Etienne Jouvin <la...@gmail.com>
>> a écrit :
>>
>>> Mark,
>>>
>>> Hum fine, I was looking the source code and touch this point ;)
>>>
>>> Thanks a lot.
>>> I am going to play with that.
>>>
>>> Etienne
>>>
>>>
>>> Le lun. 10 févr. 2020 à 14:55, Mark Payne <ma...@hotmail.com> a
>>> écrit :
>>>
>>>> Etienne,
>>>>
>>>> When a FlowFile fails, ExecuteSQL penalizes the FlowFile. This allows
>>>> you to loop failures without constantly hitting the database. By default,
>>>> the FlowFile will be penalized for 60 seconds. See [1] for more information
>>>> on how penalization works and how to configure the penalty duration.
>>>>
>>>> Thanks
>>>> -Mark
>>>>
>>>> [1]
>>>> http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#settings-tab
>>>>
>>>> > On Feb 10, 2020, at 8:47 AM, Etienne Jouvin <la...@gmail.com>
>>>> wrote:
>>>> >
>>>> > Hello All.
>>>> >
>>>> > Here is an extract of my process
>>>> > <image.png>
>>>> >
>>>> > If the executeSQL failed (invalid SQL for example), the flowfile goes
>>>> to the failure relation.
>>>> > At very first, I call the update attribute and I saw that the
>>>> flowfile is kept into the relation, never proceeded.
>>>> > So I try to put intermediate processor, LogMessage, but this is the
>>>> same.
>>>> >
>>>> > Notice that I do not have this in the success relation.
>>>> >
>>>> > Does someone have this also ?
>>>> >
>>>> > Regards
>>>> >
>>>> > Etienne Jouvin
>>>> >
>>>>
>>>>
Re: On ExecuteSQL (1.9.2) failure, infinite loop
Posted by Pierre Villard <pi...@gmail.com>.
Etienne,
The penalty duration is particularly useful when you have the relationship
going back to the ExecuteSQL processor (self-loop). In that case, you don't
want to constantly hit the database and give some time before trying again.
HTH,
Pierre
Le lun. 10 févr. 2020 à 06:03, Etienne Jouvin <la...@gmail.com> a
écrit :
> Put 0 seconds to penalized and it goes to the logger processor without
> waiting.
>
> That's fine.
>
> I just do not understand weel why there is a penalty or yield failure, but
> now that I know this, this is ok.
>
>
> Le lun. 10 févr. 2020 à 14:58, Etienne Jouvin <la...@gmail.com> a
> écrit :
>
>> Mark,
>>
>> Hum fine, I was looking the source code and touch this point ;)
>>
>> Thanks a lot.
>> I am going to play with that.
>>
>> Etienne
>>
>>
>> Le lun. 10 févr. 2020 à 14:55, Mark Payne <ma...@hotmail.com> a
>> écrit :
>>
>>> Etienne,
>>>
>>> When a FlowFile fails, ExecuteSQL penalizes the FlowFile. This allows
>>> you to loop failures without constantly hitting the database. By default,
>>> the FlowFile will be penalized for 60 seconds. See [1] for more information
>>> on how penalization works and how to configure the penalty duration.
>>>
>>> Thanks
>>> -Mark
>>>
>>> [1]
>>> http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#settings-tab
>>>
>>> > On Feb 10, 2020, at 8:47 AM, Etienne Jouvin <la...@gmail.com>
>>> wrote:
>>> >
>>> > Hello All.
>>> >
>>> > Here is an extract of my process
>>> > <image.png>
>>> >
>>> > If the executeSQL failed (invalid SQL for example), the flowfile goes
>>> to the failure relation.
>>> > At very first, I call the update attribute and I saw that the flowfile
>>> is kept into the relation, never proceeded.
>>> > So I try to put intermediate processor, LogMessage, but this is the
>>> same.
>>> >
>>> > Notice that I do not have this in the success relation.
>>> >
>>> > Does someone have this also ?
>>> >
>>> > Regards
>>> >
>>> > Etienne Jouvin
>>> >
>>>
>>>
Re: On ExecuteSQL (1.9.2) failure, infinite loop
Posted by Etienne Jouvin <la...@gmail.com>.
Put 0 seconds to penalized and it goes to the logger processor without
waiting.
That's fine.
I just do not understand weel why there is a penalty or yield failure, but
now that I know this, this is ok.
Le lun. 10 févr. 2020 à 14:58, Etienne Jouvin <la...@gmail.com> a
écrit :
> Mark,
>
> Hum fine, I was looking the source code and touch this point ;)
>
> Thanks a lot.
> I am going to play with that.
>
> Etienne
>
>
> Le lun. 10 févr. 2020 à 14:55, Mark Payne <ma...@hotmail.com> a écrit :
>
>> Etienne,
>>
>> When a FlowFile fails, ExecuteSQL penalizes the FlowFile. This allows you
>> to loop failures without constantly hitting the database. By default, the
>> FlowFile will be penalized for 60 seconds. See [1] for more information on
>> how penalization works and how to configure the penalty duration.
>>
>> Thanks
>> -Mark
>>
>> [1]
>> http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#settings-tab
>>
>> > On Feb 10, 2020, at 8:47 AM, Etienne Jouvin <la...@gmail.com>
>> wrote:
>> >
>> > Hello All.
>> >
>> > Here is an extract of my process
>> > <image.png>
>> >
>> > If the executeSQL failed (invalid SQL for example), the flowfile goes
>> to the failure relation.
>> > At very first, I call the update attribute and I saw that the flowfile
>> is kept into the relation, never proceeded.
>> > So I try to put intermediate processor, LogMessage, but this is the
>> same.
>> >
>> > Notice that I do not have this in the success relation.
>> >
>> > Does someone have this also ?
>> >
>> > Regards
>> >
>> > Etienne Jouvin
>> >
>>
>>
Re: On ExecuteSQL (1.9.2) failure, infinite loop
Posted by Etienne Jouvin <la...@gmail.com>.
Mark,
Hum fine, I was looking the source code and touch this point ;)
Thanks a lot.
I am going to play with that.
Etienne
Le lun. 10 févr. 2020 à 14:55, Mark Payne <ma...@hotmail.com> a écrit :
> Etienne,
>
> When a FlowFile fails, ExecuteSQL penalizes the FlowFile. This allows you
> to loop failures without constantly hitting the database. By default, the
> FlowFile will be penalized for 60 seconds. See [1] for more information on
> how penalization works and how to configure the penalty duration.
>
> Thanks
> -Mark
>
> [1]
> http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#settings-tab
>
> > On Feb 10, 2020, at 8:47 AM, Etienne Jouvin <la...@gmail.com>
> wrote:
> >
> > Hello All.
> >
> > Here is an extract of my process
> > <image.png>
> >
> > If the executeSQL failed (invalid SQL for example), the flowfile goes to
> the failure relation.
> > At very first, I call the update attribute and I saw that the flowfile
> is kept into the relation, never proceeded.
> > So I try to put intermediate processor, LogMessage, but this is the same.
> >
> > Notice that I do not have this in the success relation.
> >
> > Does someone have this also ?
> >
> > Regards
> >
> > Etienne Jouvin
> >
>
>
Re: On ExecuteSQL (1.9.2) failure, infinite loop
Posted by Mark Payne <ma...@hotmail.com>.
Etienne,
When a FlowFile fails, ExecuteSQL penalizes the FlowFile. This allows you to loop failures without constantly hitting the database. By default, the FlowFile will be penalized for 60 seconds. See [1] for more information on how penalization works and how to configure the penalty duration.
Thanks
-Mark
[1] http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#settings-tab
> On Feb 10, 2020, at 8:47 AM, Etienne Jouvin <la...@gmail.com> wrote:
>
> Hello All.
>
> Here is an extract of my process
> <image.png>
>
> If the executeSQL failed (invalid SQL for example), the flowfile goes to the failure relation.
> At very first, I call the update attribute and I saw that the flowfile is kept into the relation, never proceeded.
> So I try to put intermediate processor, LogMessage, but this is the same.
>
> Notice that I do not have this in the success relation.
>
> Does someone have this also ?
>
> Regards
>
> Etienne Jouvin
>
Re: On ExecuteSQL (1.9.2) failure, infinite loop
Posted by Etienne Jouvin <la...@gmail.com>.
Screen capture as linked file.
But this is pretty "simple".
ExecuteSQL processor, with a failure relation to logmessage.
By the way, the query is a "simple" select, but with something wrong inside.
It is working on an H2 database.
Le lun. 10 févr. 2020 à 14:50, Joe Witt <jo...@gmail.com> a écrit :
> Hello
>
> I cannot quite gather what you're observing from your description or the
> image. Can you send an image that is a bit zoomed out to show the full
> picture of the flow that is impacted?
>
> Thanks
>
> On Mon, Feb 10, 2020 at 5:48 AM Etienne Jouvin <la...@gmail.com>
> wrote:
>
>> Hello All.
>>
>> Here is an extract of my process
>> [image: image.png]
>>
>> If the executeSQL failed (invalid SQL for example), the flowfile goes to
>> the failure relation.
>> At very first, I call the update attribute and I saw that the flowfile is
>> kept into the relation, never proceeded.
>> So I try to put intermediate processor, LogMessage, but this is the same.
>>
>> Notice that I do not have this in the success relation.
>>
>> Does someone have this also ?
>>
>> Regards
>>
>> Etienne Jouvin
>>
>>
Re: On ExecuteSQL (1.9.2) failure, infinite loop
Posted by Joe Witt <jo...@gmail.com>.
Hello
I cannot quite gather what you're observing from your description or the
image. Can you send an image that is a bit zoomed out to show the full
picture of the flow that is impacted?
Thanks
On Mon, Feb 10, 2020 at 5:48 AM Etienne Jouvin <la...@gmail.com>
wrote:
> Hello All.
>
> Here is an extract of my process
> [image: image.png]
>
> If the executeSQL failed (invalid SQL for example), the flowfile goes to
> the failure relation.
> At very first, I call the update attribute and I saw that the flowfile is
> kept into the relation, never proceeded.
> So I try to put intermediate processor, LogMessage, but this is the same.
>
> Notice that I do not have this in the success relation.
>
> Does someone have this also ?
>
> Regards
>
> Etienne Jouvin
>
>