You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Joe Witt <jo...@gmail.com> on 2015/09/22 12:32:47 UTC
Re: How to get upstream connections or parent Processors
Hello
Addressing Grace's question:
What you are looking for is 'context' and the way context is included
is on attributes of a given FlowFile. You can, for example, use
UpdateAttribute to set that context in the proper part of the flow
preceding your custom processor.
Addressing Rick's comment:
I think we mentioned we could add a way to annotate relationships on
processors as being auto-terminated by default. Not aware of a JIRA
for this yet though. It simply provides a way for the processor to
developer to signal the default behavior is that a relationship is
unused and that is ok vs our current approach which says each
relationship must be chosen as terminated or connected.
Thanks
Joe
On Mon, Sep 21, 2015 at 11:26 PM, Rick Braddy <rb...@softnas.com> wrote:
> Me, too.
>
> And I need the ability to determine which connection(s) are linked to each relationship, along with a way to auto-terminate unused relationships from my processor.
>
> Thanks
> Rick
>
>> On Sep 21, 2015, at 10:22 PM, Huang, Jie <ji...@intel.com> wrote:
>>
>> Hi all,
>>
>> I need to get the connection information or parents for further action in onTriger event.
>> Is there any way to get the upstream connections or parent processors from any customized Processor?
>>
>> Thank you && Best Regards,
>> Grace （Huang Jie)
>>
Re: How to get upstream connections or parent Processors
Posted by Rick Braddy <rb...@softnas.com>.
Bryan,
Yes, this is what I need.
Also, need a call like terminateConnection( boolean autoTerminate, Relationship rel )
Need this at runtime as relationships are dynamically created.
Thank you.
Rick
> On Sep 22, 2015, at 8:06 AM, Bryan Bende <bb...@gmail.com> wrote:
>
> Not sure if this is exactly what everyone is looking for, but if you are
> working against master (0.3.1-SNAPSHOT) then the ProcessContext interface
> now has two new methods:
>
> boolean hasIncomingConnection();
> boolean hasConnection(Relationship relationship);
>
> This way a processor can check hasIncomingConnection() to know if it should
> expect incoming FlowFiles, and can check hasConnection(Relationship) to
> know if it should even bother transferring a FlowFile to an outgoing
> relationship.
> These were added in support of making ExecuteSQL work with and without
> incoming FlowFiles [1].
>
> -Bryan
>
> [1] https://issues.apache.org/jira/browse/NIFI-932
>
>
>> On Tue, Sep 22, 2015 at 6:32 AM, Joe Witt <jo...@gmail.com> wrote:
>>
>> Hello
>>
>> Addressing Grace's question:
>>
>> What you are looking for is 'context' and the way context is included
>> is on attributes of a given FlowFile. You can, for example, use
>> UpdateAttribute to set that context in the proper part of the flow
>> preceding your custom processor.
>>
>> Addressing Rick's comment:
>>
>> I think we mentioned we could add a way to annotate relationships on
>> processors as being auto-terminated by default. Not aware of a JIRA
>> for this yet though. It simply provides a way for the processor to
>> developer to signal the default behavior is that a relationship is
>> unused and that is ok vs our current approach which says each
>> relationship must be chosen as terminated or connected.
>>
>> Thanks
>> Joe
>>
>>> On Mon, Sep 21, 2015 at 11:26 PM, Rick Braddy <rb...@softnas.com> wrote:
>>> Me, too.
>>>
>>> And I need the ability to determine which connection(s) are linked to
>> each relationship, along with a way to auto-terminate unused relationships
>> from my processor.
>>>
>>> Thanks
>>> Rick
>>>
>>>> On Sep 21, 2015, at 10:22 PM, Huang, Jie <ji...@intel.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I need to get the connection information or parents for further action
>> in onTriger event.
>>>> Is there any way to get the upstream connections or parent processors
>> from any customized Processor?
>>>>
>>>> Thank you && Best Regards,
>>>> Grace （Huang Jie)
>>
Re: How to get upstream connections or parent Processors
Posted by Rick Braddy <rb...@softnas.com>.
Yeah. I agree and only want to auto terminate at runtime in certain cases and not as a general rule or by default.
> On Sep 22, 2015, at 8:22 AM, Corey Flowers <cf...@onyxpoint.com> wrote:
>
> I would be a little weary of auto terminating relationships that aren't
> defined. I would expect that to lead to unwanted data loss. If that is
> implemented, can we make that a configuration option or something? I would
> not want that to be the default in my graphs for sure.
>
>> On Tue, Sep 22, 2015 at 9:05 AM, Bryan Bende <bb...@gmail.com> wrote:
>>
>> Not sure if this is exactly what everyone is looking for, but if you are
>> working against master (0.3.1-SNAPSHOT) then the ProcessContext interface
>> now has two new methods:
>>
>> boolean hasIncomingConnection();
>> boolean hasConnection(Relationship relationship);
>>
>> This way a processor can check hasIncomingConnection() to know if it should
>> expect incoming FlowFiles, and can check hasConnection(Relationship) to
>> know if it should even bother transferring a FlowFile to an outgoing
>> relationship.
>> These were added in support of making ExecuteSQL work with and without
>> incoming FlowFiles [1].
>>
>> -Bryan
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-932
>>
>>
>>> On Tue, Sep 22, 2015 at 6:32 AM, Joe Witt <jo...@gmail.com> wrote:
>>>
>>> Hello
>>>
>>> Addressing Grace's question:
>>>
>>> What you are looking for is 'context' and the way context is included
>>> is on attributes of a given FlowFile. You can, for example, use
>>> UpdateAttribute to set that context in the proper part of the flow
>>> preceding your custom processor.
>>>
>>> Addressing Rick's comment:
>>>
>>> I think we mentioned we could add a way to annotate relationships on
>>> processors as being auto-terminated by default. Not aware of a JIRA
>>> for this yet though. It simply provides a way for the processor to
>>> developer to signal the default behavior is that a relationship is
>>> unused and that is ok vs our current approach which says each
>>> relationship must be chosen as terminated or connected.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Mon, Sep 21, 2015 at 11:26 PM, Rick Braddy <rb...@softnas.com>
>> wrote:
>>>> Me, too.
>>>>
>>>> And I need the ability to determine which connection(s) are linked to
>>> each relationship, along with a way to auto-terminate unused
>> relationships
>>> from my processor.
>>>>
>>>> Thanks
>>>> Rick
>>>>
>>>>> On Sep 21, 2015, at 10:22 PM, Huang, Jie <ji...@intel.com> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I need to get the connection information or parents for further action
>>> in onTriger event.
>>>>> Is there any way to get the upstream connections or parent processors
>>> from any customized Processor?
>>>>>
>>>>> Thank you && Best Regards,
>>>>> Grace （Huang Jie)
>
>
>
> --
> Corey Flowers
> Vice President, Onyx Point, Inc
> (410) 541-6699
> cflowers@onyxpoint.com
>
> -- This account not approved for unencrypted proprietary information --
Re: How to get upstream connections or parent Processors
Posted by Corey Flowers <cf...@onyxpoint.com>.
I would be a little weary of auto terminating relationships that aren't
defined. I would expect that to lead to unwanted data loss. If that is
implemented, can we make that a configuration option or something? I would
not want that to be the default in my graphs for sure.
On Tue, Sep 22, 2015 at 9:05 AM, Bryan Bende <bb...@gmail.com> wrote:
> Not sure if this is exactly what everyone is looking for, but if you are
> working against master (0.3.1-SNAPSHOT) then the ProcessContext interface
> now has two new methods:
>
> boolean hasIncomingConnection();
> boolean hasConnection(Relationship relationship);
>
> This way a processor can check hasIncomingConnection() to know if it should
> expect incoming FlowFiles, and can check hasConnection(Relationship) to
> know if it should even bother transferring a FlowFile to an outgoing
> relationship.
> These were added in support of making ExecuteSQL work with and without
> incoming FlowFiles [1].
>
> -Bryan
>
> [1] https://issues.apache.org/jira/browse/NIFI-932
>
>
> On Tue, Sep 22, 2015 at 6:32 AM, Joe Witt <jo...@gmail.com> wrote:
>
> > Hello
> >
> > Addressing Grace's question:
> >
> > What you are looking for is 'context' and the way context is included
> > is on attributes of a given FlowFile. You can, for example, use
> > UpdateAttribute to set that context in the proper part of the flow
> > preceding your custom processor.
> >
> > Addressing Rick's comment:
> >
> > I think we mentioned we could add a way to annotate relationships on
> > processors as being auto-terminated by default. Not aware of a JIRA
> > for this yet though. It simply provides a way for the processor to
> > developer to signal the default behavior is that a relationship is
> > unused and that is ok vs our current approach which says each
> > relationship must be chosen as terminated or connected.
> >
> > Thanks
> > Joe
> >
> > On Mon, Sep 21, 2015 at 11:26 PM, Rick Braddy <rb...@softnas.com>
> wrote:
> > > Me, too.
> > >
> > > And I need the ability to determine which connection(s) are linked to
> > each relationship, along with a way to auto-terminate unused
> relationships
> > from my processor.
> > >
> > > Thanks
> > > Rick
> > >
> > >> On Sep 21, 2015, at 10:22 PM, Huang, Jie <ji...@intel.com> wrote:
> > >>
> > >> Hi all,
> > >>
> > >> I need to get the connection information or parents for further action
> > in onTriger event.
> > >> Is there any way to get the upstream connections or parent processors
> > from any customized Processor?
> > >>
> > >> Thank you && Best Regards,
> > >> Grace （Huang Jie)
> > >>
> >
>
--
Corey Flowers
Vice President, Onyx Point, Inc
(410) 541-6699
cflowers@onyxpoint.com
-- This account not approved for unencrypted proprietary information --
Re: How to get upstream connections or parent Processors
Posted by Bryan Bende <bb...@gmail.com>.
Not sure if this is exactly what everyone is looking for, but if you are
working against master (0.3.1-SNAPSHOT) then the ProcessContext interface
now has two new methods:
boolean hasIncomingConnection();
boolean hasConnection(Relationship relationship);
This way a processor can check hasIncomingConnection() to know if it should
expect incoming FlowFiles, and can check hasConnection(Relationship) to
know if it should even bother transferring a FlowFile to an outgoing
relationship.
These were added in support of making ExecuteSQL work with and without
incoming FlowFiles [1].
-Bryan
[1] https://issues.apache.org/jira/browse/NIFI-932
On Tue, Sep 22, 2015 at 6:32 AM, Joe Witt <jo...@gmail.com> wrote:
> Hello
>
> Addressing Grace's question:
>
> What you are looking for is 'context' and the way context is included
> is on attributes of a given FlowFile. You can, for example, use
> UpdateAttribute to set that context in the proper part of the flow
> preceding your custom processor.
>
> Addressing Rick's comment:
>
> I think we mentioned we could add a way to annotate relationships on
> processors as being auto-terminated by default. Not aware of a JIRA
> for this yet though. It simply provides a way for the processor to
> developer to signal the default behavior is that a relationship is
> unused and that is ok vs our current approach which says each
> relationship must be chosen as terminated or connected.
>
> Thanks
> Joe
>
> On Mon, Sep 21, 2015 at 11:26 PM, Rick Braddy <rb...@softnas.com> wrote:
> > Me, too.
> >
> > And I need the ability to determine which connection(s) are linked to
> each relationship, along with a way to auto-terminate unused relationships
> from my processor.
> >
> > Thanks
> > Rick
> >
> >> On Sep 21, 2015, at 10:22 PM, Huang, Jie <ji...@intel.com> wrote:
> >>
> >> Hi all,
> >>
> >> I need to get the connection information or parents for further action
> in onTriger event.
> >> Is there any way to get the upstream connections or parent processors
> from any customized Processor?
> >>
> >> Thank you && Best Regards,
> >> Grace （Huang Jie)
> >>
>