You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Mike Thomsen <mi...@gmail.com> on 2018/12/20 16:45:01 UTC
Checking for existence of mandatory attributes
Do we have any processors that would be particularly good at checking a
flowfile for the existence of certain mandatory attributes and then routing
flowfiles missing them to a failure relationship?
I have a processor sketched out in my head for this already, but don't want
to reinvent the wheel.
Thanks,
Mike
Re: Checking for existence of mandatory attributes
Posted by Bryan Bende <bb...@gmail.com>.
RouteOnAttribute?
On Thu, Dec 20, 2018 at 11:45 AM Mike Thomsen <mi...@gmail.com> wrote:
>
> Do we have any processors that would be particularly good at checking a
> flowfile for the existence of certain mandatory attributes and then routing
> flowfiles missing them to a failure relationship?
>
> I have a processor sketched out in my head for this already, but don't want
> to reinvent the wheel.
>
> Thanks,
>
> Mike
Re: Checking for existence of mandatory attributes
Posted by Mike Thomsen <mi...@gmail.com>.
I ended up getting a little creative by chaining a few Puts with multiple
RouteOnAttributes. Worked quite well so far.
On Thu, Dec 20, 2018 at 1:59 PM Mark Payne <ma...@hotmail.com> wrote:
> Mike,
>
> I do think that RouteOnAttribute is the correct play here, given the
> details at hand.
> Other options, should that be too cumbersome to configure may be
> UpdateAttribute using the
> Advanced configuration, which works sort of like a mini rules engine. Or,
> to be honest,
> this sounds like it could be a great use of ExecuteScript.
>
> If you want to create a new Processor, then by all means do so, but I
> think we'd need a better
> description of the use case for it and description of where the other
> approaches fall short and how this new
> Processor would fill in the gap in order for it to be warranted as a
> standard processor.
>
> Thanks
> -Mark
>
> > On Dec 20, 2018, at 12:19 PM, Mike Thomsen <mi...@gmail.com>
> wrote:
> >
> > I might still hack out a prototype and send it as a PR because it would
> be
> > like a really beefed up version of RouteOnAttribute that is built around
> > policy enforcement with attributes.
> >
> > On Thu, Dec 20, 2018 at 12:18 PM Mike Thomsen <mi...@gmail.com>
> > wrote:
> >
> >> This is probably enough:
> >>
> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#allattributes
> >>
> >> I missed that because I was thinking "how do I do a bunch of 'and must
> >> exist' statements."
> >>
> >> Thanks,
> >>
> >> Mike
> >>
> >> On Thu, Dec 20, 2018 at 12:01 PM Mark Payne <ma...@hotmail.com>
> wrote:
> >>
> >>> Hey Mike,
> >>>
> >>> I would recommend RouteOnAttribute for that.
> >>>
> >>> Thanks
> >>> -Mark
> >>>
> >>> Sent from my iPhone
> >>>
> >>>> On Dec 20, 2018, at 11:45 AM, Mike Thomsen <mi...@gmail.com>
> >>> wrote:
> >>>>
> >>>> Do we have any processors that would be particularly good at checking
> a
> >>>> flowfile for the existence of certain mandatory attributes and then
> >>> routing
> >>>> flowfiles missing them to a failure relationship?
> >>>>
> >>>> I have a processor sketched out in my head for this already, but don't
> >>> want
> >>>> to reinvent the wheel.
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Mike
> >>>
> >>
>
>
Re: Checking for existence of mandatory attributes
Posted by Mark Payne <ma...@hotmail.com>.
Mike,
I do think that RouteOnAttribute is the correct play here, given the details at hand.
Other options, should that be too cumbersome to configure may be UpdateAttribute using the
Advanced configuration, which works sort of like a mini rules engine. Or, to be honest,
this sounds like it could be a great use of ExecuteScript.
If you want to create a new Processor, then by all means do so, but I think we'd need a better
description of the use case for it and description of where the other approaches fall short and how this new
Processor would fill in the gap in order for it to be warranted as a standard processor.
Thanks
-Mark
> On Dec 20, 2018, at 12:19 PM, Mike Thomsen <mi...@gmail.com> wrote:
>
> I might still hack out a prototype and send it as a PR because it would be
> like a really beefed up version of RouteOnAttribute that is built around
> policy enforcement with attributes.
>
> On Thu, Dec 20, 2018 at 12:18 PM Mike Thomsen <mi...@gmail.com>
> wrote:
>
>> This is probably enough:
>> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#allattributes
>>
>> I missed that because I was thinking "how do I do a bunch of 'and must
>> exist' statements."
>>
>> Thanks,
>>
>> Mike
>>
>> On Thu, Dec 20, 2018 at 12:01 PM Mark Payne <ma...@hotmail.com> wrote:
>>
>>> Hey Mike,
>>>
>>> I would recommend RouteOnAttribute for that.
>>>
>>> Thanks
>>> -Mark
>>>
>>> Sent from my iPhone
>>>
>>>> On Dec 20, 2018, at 11:45 AM, Mike Thomsen <mi...@gmail.com>
>>> wrote:
>>>>
>>>> Do we have any processors that would be particularly good at checking a
>>>> flowfile for the existence of certain mandatory attributes and then
>>> routing
>>>> flowfiles missing them to a failure relationship?
>>>>
>>>> I have a processor sketched out in my head for this already, but don't
>>> want
>>>> to reinvent the wheel.
>>>>
>>>> Thanks,
>>>>
>>>> Mike
>>>
>>
Re: Checking for existence of mandatory attributes
Posted by Mike Thomsen <mi...@gmail.com>.
I might still hack out a prototype and send it as a PR because it would be
like a really beefed up version of RouteOnAttribute that is built around
policy enforcement with attributes.
On Thu, Dec 20, 2018 at 12:18 PM Mike Thomsen <mi...@gmail.com>
wrote:
> This is probably enough:
> https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#allattributes
>
> I missed that because I was thinking "how do I do a bunch of 'and must
> exist' statements."
>
> Thanks,
>
> Mike
>
> On Thu, Dec 20, 2018 at 12:01 PM Mark Payne <ma...@hotmail.com> wrote:
>
>> Hey Mike,
>>
>> I would recommend RouteOnAttribute for that.
>>
>> Thanks
>> -Mark
>>
>> Sent from my iPhone
>>
>> > On Dec 20, 2018, at 11:45 AM, Mike Thomsen <mi...@gmail.com>
>> wrote:
>> >
>> > Do we have any processors that would be particularly good at checking a
>> > flowfile for the existence of certain mandatory attributes and then
>> routing
>> > flowfiles missing them to a failure relationship?
>> >
>> > I have a processor sketched out in my head for this already, but don't
>> want
>> > to reinvent the wheel.
>> >
>> > Thanks,
>> >
>> > Mike
>>
>
Re: Checking for existence of mandatory attributes
Posted by Mike Thomsen <mi...@gmail.com>.
This is probably enough:
https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#allattributes
I missed that because I was thinking "how do I do a bunch of 'and must
exist' statements."
Thanks,
Mike
On Thu, Dec 20, 2018 at 12:01 PM Mark Payne <ma...@hotmail.com> wrote:
> Hey Mike,
>
> I would recommend RouteOnAttribute for that.
>
> Thanks
> -Mark
>
> Sent from my iPhone
>
> > On Dec 20, 2018, at 11:45 AM, Mike Thomsen <mi...@gmail.com>
> wrote:
> >
> > Do we have any processors that would be particularly good at checking a
> > flowfile for the existence of certain mandatory attributes and then
> routing
> > flowfiles missing them to a failure relationship?
> >
> > I have a processor sketched out in my head for this already, but don't
> want
> > to reinvent the wheel.
> >
> > Thanks,
> >
> > Mike
>
Re: Checking for existence of mandatory attributes
Posted by Mark Payne <ma...@hotmail.com>.
Hey Mike,
I would recommend RouteOnAttribute for that.
Thanks
-Mark
Sent from my iPhone
> On Dec 20, 2018, at 11:45 AM, Mike Thomsen <mi...@gmail.com> wrote:
>
> Do we have any processors that would be particularly good at checking a
> flowfile for the existence of certain mandatory attributes and then routing
> flowfiles missing them to a failure relationship?
>
> I have a processor sketched out in my head for this already, but don't want
> to reinvent the wheel.
>
> Thanks,
>
> Mike