You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Lars Francke <la...@gmail.com> on 2016/01/10 21:32:08 UTC
One-time Processors
Hi,
I'm just getting started with Nifi and understand that I probably still
have some misconceptions :)
For various reasons I'd love to use NiFi for one-time actions/processors
which I'd like to trigger as needed from the UI. I've seen the various
scheduling strategies and the closest I've come is to have a really long
run schedule or a specific cron schedule.
What I'd really like is a way for a Processor to gracefully stop itself
when it's done processing.
Is there any way to achieve this?
Thanks for your help
Re: One-time Processors
Posted by Matthew Clarke <ma...@gmail.com>.
Lars,
If start a processor and then stop it, the thread does not get killed.
The processor will continue to run until that thread completes its job. The
processor will then remain stopped and not execute again until manually
started again.
Matt
On Jan 18, 2016 10:54 AM, "Joe Percivall" <jo...@yahoo.com> wrote:
> Joe and Lars,
>
> I created a Jira ticket for this issue:
> https://issues.apache.org/jira/browse/NIFI-1407
>
> We can continue the conversation there instead two different email threads.
>
> Joe
> - - - - - -
> *Joseph Percivall*
> linkedin.com/in/Percivall
> e: joepercivall@yahoo.com
>
>
>
> On Sunday, January 10, 2016 4:03 PM, Lars Francke <la...@gmail.com>
> wrote:
>
>
> Joe,
>
> thanks for the welcome and the (really quick) reply.
>
> My use-case (and again I may use NiFi for things it wasn't intended to) is
> for a one-time/initial-import scenario. I have a Processor that I'd like to
> start that then calls a Web service and pages through all results creating
> FlowFiles and transferring them onwards. When it's reached the end I'd like
> it to Stop automatically without being rescheduled.
>
> I'd like to do it this way to maximise code-reuse with other flows (that
> for example do incremental imports, a reason I'm excited about the upcoming
> State Management) and get the benefits of monitoring etc.
>
> So the feature proposed would work for me (run X times). Has a JIRA for
> this been filed that I can track (looked but I couldn't find one)?
> If I understand correctly there's currently no way from within a Processor
> to stop itself?
>
> Cheers,
> Lars
>
>
>
> On Sun, Jan 10, 2016 at 9:37 PM, Joe Witt <jo...@gmail.com> wrote:
>
> Lars,
>
> First, welcome to NiFi.
>
> So you'll initiate the flow from NiFi through some manual trigger and
> that part makes sense/sounds straightforward. But then the question
> is 'once the flow is done' I'd like it to shut down. What are the
> processors/things that begin the flow? We've recently had someone
> suggest adding a way to tell a processor to only run 'X' times before
> being unscheduled and thus requiring manual start. Sounds like that
> might help you with your case as well.
>
> There is little impact to having all the processors after the first
> one running. If there is no work to do they'll be quite efficient.
>
> Do you think if a processor had a 'max executions' concept that it
> would address your need?
>
> Thanks
> Joe
>
> On Sun, Jan 10, 2016 at 3:32 PM, Lars Francke <la...@gmail.com>
> wrote:
> > Hi,
> >
> > I'm just getting started with Nifi and understand that I probably still
> have
> > some misconceptions :)
> >
> > For various reasons I'd love to use NiFi for one-time actions/processors
> > which I'd like to trigger as needed from the UI. I've seen the various
> > scheduling strategies and the closest I've come is to have a really long
> run
> > schedule or a specific cron schedule.
> >
> > What I'd really like is a way for a Processor to gracefully stop itself
> when
> > it's done processing.
> >
> > Is there any way to achieve this?
> >
> > Thanks for your help
>
>
>
>
>
Re: One-time Processors
Posted by Joe Percivall <jo...@yahoo.com>.
Joe and Lars,
I created a Jira ticket for this issue: https://issues.apache.org/jira/browse/NIFI-1407
We can continue the conversation there instead two different email threads.
Joe - - - - - - Joseph Percivalllinkedin.com/in/Percivalle: joepercivall@yahoo.com
On Sunday, January 10, 2016 4:03 PM, Lars Francke <la...@gmail.com> wrote:
Joe,
thanks for the welcome and the (really quick) reply.
My use-case (and again I may use NiFi for things it wasn't intended to) is for a one-time/initial-import scenario. I have a Processor that I'd like to start that then calls a Web service and pages through all results creating FlowFiles and transferring them onwards. When it's reached the end I'd like it to Stop automatically without being rescheduled.
I'd like to do it this way to maximise code-reuse with other flows (that for example do incremental imports, a reason I'm excited about the upcoming State Management) and get the benefits of monitoring etc.
So the feature proposed would work for me (run X times). Has a JIRA for this been filed that I can track (looked but I couldn't find one)?If I understand correctly there's currently no way from within a Processor to stop itself?
Cheers,Lars
On Sun, Jan 10, 2016 at 9:37 PM, Joe Witt <jo...@gmail.com> wrote:
Lars,
First, welcome to NiFi.
So you'll initiate the flow from NiFi through some manual trigger and
that part makes sense/sounds straightforward. But then the question
is 'once the flow is done' I'd like it to shut down. What are the
processors/things that begin the flow? We've recently had someone
suggest adding a way to tell a processor to only run 'X' times before
being unscheduled and thus requiring manual start. Sounds like that
might help you with your case as well.
There is little impact to having all the processors after the first
one running. If there is no work to do they'll be quite efficient.
Do you think if a processor had a 'max executions' concept that it
would address your need?
Thanks
Joe
On Sun, Jan 10, 2016 at 3:32 PM, Lars Francke <la...@gmail.com> wrote:
> Hi,
>
> I'm just getting started with Nifi and understand that I probably still have
> some misconceptions :)
>
> For various reasons I'd love to use NiFi for one-time actions/processors
> which I'd like to trigger as needed from the UI. I've seen the various
> scheduling strategies and the closest I've come is to have a really long run
> schedule or a specific cron schedule.
>
> What I'd really like is a way for a Processor to gracefully stop itself when
> it's done processing.
>
> Is there any way to achieve this?
>
> Thanks for your help
Re: One-time Processors
Posted by Lars Francke <la...@gmail.com>.
Joe,
thanks for the welcome and the (really quick) reply.
My use-case (and again I may use NiFi for things it wasn't intended to) is
for a one-time/initial-import scenario. I have a Processor that I'd like to
start that then calls a Web service and pages through all results creating
FlowFiles and transferring them onwards. When it's reached the end I'd like
it to Stop automatically without being rescheduled.
I'd like to do it this way to maximise code-reuse with other flows (that
for example do incremental imports, a reason I'm excited about the upcoming
State Management) and get the benefits of monitoring etc.
So the feature proposed would work for me (run X times). Has a JIRA for
this been filed that I can track (looked but I couldn't find one)?
If I understand correctly there's currently no way from within a Processor
to stop itself?
Cheers,
Lars
On Sun, Jan 10, 2016 at 9:37 PM, Joe Witt <jo...@gmail.com> wrote:
> Lars,
>
> First, welcome to NiFi.
>
> So you'll initiate the flow from NiFi through some manual trigger and
> that part makes sense/sounds straightforward. But then the question
> is 'once the flow is done' I'd like it to shut down. What are the
> processors/things that begin the flow? We've recently had someone
> suggest adding a way to tell a processor to only run 'X' times before
> being unscheduled and thus requiring manual start. Sounds like that
> might help you with your case as well.
>
> There is little impact to having all the processors after the first
> one running. If there is no work to do they'll be quite efficient.
>
> Do you think if a processor had a 'max executions' concept that it
> would address your need?
>
> Thanks
> Joe
>
> On Sun, Jan 10, 2016 at 3:32 PM, Lars Francke <la...@gmail.com>
> wrote:
> > Hi,
> >
> > I'm just getting started with Nifi and understand that I probably still
> have
> > some misconceptions :)
> >
> > For various reasons I'd love to use NiFi for one-time actions/processors
> > which I'd like to trigger as needed from the UI. I've seen the various
> > scheduling strategies and the closest I've come is to have a really long
> run
> > schedule or a specific cron schedule.
> >
> > What I'd really like is a way for a Processor to gracefully stop itself
> when
> > it's done processing.
> >
> > Is there any way to achieve this?
> >
> > Thanks for your help
>
Re: One-time Processors
Posted by Joe Witt <jo...@gmail.com>.
Lars,
First, welcome to NiFi.
So you'll initiate the flow from NiFi through some manual trigger and
that part makes sense/sounds straightforward. But then the question
is 'once the flow is done' I'd like it to shut down. What are the
processors/things that begin the flow? We've recently had someone
suggest adding a way to tell a processor to only run 'X' times before
being unscheduled and thus requiring manual start. Sounds like that
might help you with your case as well.
There is little impact to having all the processors after the first
one running. If there is no work to do they'll be quite efficient.
Do you think if a processor had a 'max executions' concept that it
would address your need?
Thanks
Joe
On Sun, Jan 10, 2016 at 3:32 PM, Lars Francke <la...@gmail.com> wrote:
> Hi,
>
> I'm just getting started with Nifi and understand that I probably still have
> some misconceptions :)
>
> For various reasons I'd love to use NiFi for one-time actions/processors
> which I'd like to trigger as needed from the UI. I've seen the various
> scheduling strategies and the closest I've come is to have a really long run
> schedule or a specific cron schedule.
>
> What I'd really like is a way for a Processor to gracefully stop itself when
> it's done processing.
>
> Is there any way to achieve this?
>
> Thanks for your help