You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Brian Ghigiarelli <br...@gmail.com> on 2015/04/15 20:04:23 UTC
NiFi Queues FlowFile Expiration
Hi all,
In regards to the FlowFile expiration setting on queues between processors,
is that time based on:
- the arrival timestamp of the FlowFile into the queue,
- the first created / received timestamp of the FlowFile in the overall
flow, or
- something else entirely?
Thanks,
--
Brian Ghigiarelli
Re: NiFi Queues FlowFile Expiration
Posted by Jennifer Barnabee <je...@gmail.com>.
Brian,
It's based on the timestamp of when the FlowFile was first received/created
in the overall flow.
-Jenn
On Wed, Apr 15, 2015 at 2:04 PM, Brian Ghigiarelli <br...@gmail.com>
wrote:
> Hi all,
>
> In regards to the FlowFile expiration setting on queues between processors,
> is that time based on:
> - the arrival timestamp of the FlowFile into the queue,
> - the first created / received timestamp of the FlowFile in the overall
> flow, or
> - something else entirely?
>
> Thanks,
>
> --
> Brian Ghigiarelli
>
Re: NiFi Queues FlowFile Expiration
Posted by Mark Payne <ma...@hotmail.com>.
Brian,
Currently it would use the time stamp of when the clone happened. We do have the time stamp of when it was created, so we have considered changing to that but as of right now it is based on when the clone was created.
Thanks
-Mark
Sent from my iPad
> On Apr 15, 2015, at 2:01 PM, Brian Ghigiarelli <br...@gmail.com> wrote:
>
> Thanks for the responses, Corey and Jennifer.
>
> Corey - is "the timestamp of the file into the NiFi repo" set when the
> FlowFile is first created/received, or when the FlowFile is enqueued?
> Jennifer's answer indicates that it's set when the FlowFile is first
> created/received.
>
> Along these lines, if I clone and transfer a FlowFile, does it use the
> timestamp of when the FlowFile was cloned or does it use the first
> timestamp of the original FlowFile tracked by provenance?
>
> Thanks,
> Brian
>
> On Wed, Apr 15, 2015 at 2:36 PM, Corey Flowers <cf...@onyxpoint.com>
> wrote:
>
>> Good afternoon Mr. Ghigiarelli,
>>
>> The flowfile expiration is based on the timestamp of the file
>> into the Nifi repo. Not the previous processor or original age of the file.
>>
>> Hope that helps!
>>
>> Sent from my iPhone
>>
>>>> On Apr 15, 2015, at 2:04 PM, Brian Ghigiarelli <br...@gmail.com>
>>> wrote:
>>>
>>> Hi all,
>>>
>>> In regards to the FlowFile expiration setting on queues between
>> processors,
>>> is that time based on:
>>> - the arrival timestamp of the FlowFile into the queue,
>>> - the first created / received timestamp of the FlowFile in the overall
>>> flow, or
>>> - something else entirely?
>>>
>>> Thanks,
>>>
>>> --
>>> Brian Ghigiarelli
>>
Re: NiFi Queues FlowFile Expiration
Posted by Brian Ghigiarelli <br...@gmail.com>.
Great - thanks for the clarification, everybody! That's consistent with
what we're seeing while running it too.
On Wed, Apr 15, 2015 at 4:11 PM, Corey Flowers <cf...@onyxpoint.com>
wrote:
> Hey Brian,
>
> It would be the created/received time for the orig flowfile and
> like Mark said, for clones it would be the time of cloning because in
> essence a clone is a new flowfile. Jen just beat me to the punch and worded
> it a little better!
>
> Corey
>
> Sent from my iPhone
>
> > On Apr 15, 2015, at 3:00 PM, Brian Ghigiarelli <br...@gmail.com>
> wrote:
> >
> > Thanks for the responses, Corey and Jennifer.
> >
> > Corey - is "the timestamp of the file into the NiFi repo" set when the
> > FlowFile is first created/received, or when the FlowFile is enqueued?
> > Jennifer's answer indicates that it's set when the FlowFile is first
> > created/received.
> >
> > Along these lines, if I clone and transfer a FlowFile, does it use the
> > timestamp of when the FlowFile was cloned or does it use the first
> > timestamp of the original FlowFile tracked by provenance?
> >
> > Thanks,
> > Brian
> >
> > On Wed, Apr 15, 2015 at 2:36 PM, Corey Flowers <cf...@onyxpoint.com>
> > wrote:
> >
> >> Good afternoon Mr. Ghigiarelli,
> >>
> >> The flowfile expiration is based on the timestamp of the file
> >> into the Nifi repo. Not the previous processor or original age of the
> file.
> >>
> >> Hope that helps!
> >>
> >> Sent from my iPhone
> >>
> >>>> On Apr 15, 2015, at 2:04 PM, Brian Ghigiarelli <br...@gmail.com>
> >>> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> In regards to the FlowFile expiration setting on queues between
> >> processors,
> >>> is that time based on:
> >>> - the arrival timestamp of the FlowFile into the queue,
> >>> - the first created / received timestamp of the FlowFile in the overall
> >>> flow, or
> >>> - something else entirely?
> >>>
> >>> Thanks,
> >>>
> >>> --
> >>> Brian Ghigiarelli
> >>
>
Re: NiFi Queues FlowFile Expiration
Posted by Corey Flowers <cf...@onyxpoint.com>.
Hey Brian,
It would be the created/received time for the orig flowfile and like Mark said, for clones it would be the time of cloning because in essence a clone is a new flowfile. Jen just beat me to the punch and worded it a little better!
Corey
Sent from my iPhone
> On Apr 15, 2015, at 3:00 PM, Brian Ghigiarelli <br...@gmail.com> wrote:
>
> Thanks for the responses, Corey and Jennifer.
>
> Corey - is "the timestamp of the file into the NiFi repo" set when the
> FlowFile is first created/received, or when the FlowFile is enqueued?
> Jennifer's answer indicates that it's set when the FlowFile is first
> created/received.
>
> Along these lines, if I clone and transfer a FlowFile, does it use the
> timestamp of when the FlowFile was cloned or does it use the first
> timestamp of the original FlowFile tracked by provenance?
>
> Thanks,
> Brian
>
> On Wed, Apr 15, 2015 at 2:36 PM, Corey Flowers <cf...@onyxpoint.com>
> wrote:
>
>> Good afternoon Mr. Ghigiarelli,
>>
>> The flowfile expiration is based on the timestamp of the file
>> into the Nifi repo. Not the previous processor or original age of the file.
>>
>> Hope that helps!
>>
>> Sent from my iPhone
>>
>>>> On Apr 15, 2015, at 2:04 PM, Brian Ghigiarelli <br...@gmail.com>
>>> wrote:
>>>
>>> Hi all,
>>>
>>> In regards to the FlowFile expiration setting on queues between
>> processors,
>>> is that time based on:
>>> - the arrival timestamp of the FlowFile into the queue,
>>> - the first created / received timestamp of the FlowFile in the overall
>>> flow, or
>>> - something else entirely?
>>>
>>> Thanks,
>>>
>>> --
>>> Brian Ghigiarelli
>>
Re: NiFi Queues FlowFile Expiration
Posted by Brian Ghigiarelli <br...@gmail.com>.
Thanks for the responses, Corey and Jennifer.
Corey - is "the timestamp of the file into the NiFi repo" set when the
FlowFile is first created/received, or when the FlowFile is enqueued?
Jennifer's answer indicates that it's set when the FlowFile is first
created/received.
Along these lines, if I clone and transfer a FlowFile, does it use the
timestamp of when the FlowFile was cloned or does it use the first
timestamp of the original FlowFile tracked by provenance?
Thanks,
Brian
On Wed, Apr 15, 2015 at 2:36 PM, Corey Flowers <cf...@onyxpoint.com>
wrote:
> Good afternoon Mr. Ghigiarelli,
>
> The flowfile expiration is based on the timestamp of the file
> into the Nifi repo. Not the previous processor or original age of the file.
>
> Hope that helps!
>
> Sent from my iPhone
>
> > On Apr 15, 2015, at 2:04 PM, Brian Ghigiarelli <br...@gmail.com>
> wrote:
> >
> > Hi all,
> >
> > In regards to the FlowFile expiration setting on queues between
> processors,
> > is that time based on:
> > - the arrival timestamp of the FlowFile into the queue,
> > - the first created / received timestamp of the FlowFile in the overall
> > flow, or
> > - something else entirely?
> >
> > Thanks,
> >
> > --
> > Brian Ghigiarelli
>
Re: NiFi Queues FlowFile Expiration
Posted by Corey Flowers <cf...@onyxpoint.com>.
Good afternoon Mr. Ghigiarelli,
The flowfile expiration is based on the timestamp of the file into the Nifi repo. Not the previous processor or original age of the file.
Hope that helps!
Sent from my iPhone
> On Apr 15, 2015, at 2:04 PM, Brian Ghigiarelli <br...@gmail.com> wrote:
>
> Hi all,
>
> In regards to the FlowFile expiration setting on queues between processors,
> is that time based on:
> - the arrival timestamp of the FlowFile into the queue,
> - the first created / received timestamp of the FlowFile in the overall
> flow, or
> - something else entirely?
>
> Thanks,
>
> --
> Brian Ghigiarelli