You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Ricky Saltzer <ri...@cloudera.com> on 2015/03/26 18:35:25 UTC
Updating NARs on a running cluster
Hi -
I have a 3 node NiFi cluster, and periodically I need to update the custom
NAR we use for our internal projects. Currently I'm just copying the new
NAR into the lib directory on each of the nodes (including the master), and
then restarting each daemon (bin/nifi.sh restart) individually.
I'm curious if there's a better method than the one I'm using. The biggest
downside I'm seeing is that all my processors that I have scheduled to
execute once per day are being kicked off again, which is not the behavior
I want. I don't have these set to use a CRON based schedule, because there
is currently no way of having a CRON scheduled processor run on a single
node.
--
Ricky Saltzer
http://www.cloudera.com
Re: Updating NARs on a running cluster
Posted by Mike Drob <md...@apache.org>.
Alternatively, exposing rolling restarts via the cluster manager would be a
good feature to add, if it doesn't already exist.
On Thu, Mar 26, 2015 at 1:15 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Oh, I was just short-handing your description of your manual process. :)
>
> On Thu, Mar 26, 2015 at 11:10 AM, Ricky Saltzer <ri...@cloudera.com>
> wrote:
>
> > Is there a way to perform a rolling restart? Perhaps undocumented?
> >
> > $ ./bin/nifi.sh
> > Usage nifi {start|stop|run|restart|status|dump|install}
> >
> >
> > On Thu, Mar 26, 2015 at 2:01 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > If the schedule state was properly maintained across a restart, would
> the
> > > rolling-restart-to-update be fine?
> > >
> > > On Thu, Mar 26, 2015 at 10:35 AM, Ricky Saltzer <ri...@cloudera.com>
> > > wrote:
> > >
> > > > Hi -
> > > >
> > > > I have a 3 node NiFi cluster, and periodically I need to update the
> > > custom
> > > > NAR we use for our internal projects. Currently I'm just copying the
> > new
> > > > NAR into the lib directory on each of the nodes (including the
> master),
> > > and
> > > > then restarting each daemon (bin/nifi.sh restart) individually.
> > > >
> > > > I'm curious if there's a better method than the one I'm using. The
> > > biggest
> > > > downside I'm seeing is that all my processors that I have scheduled
> to
> > > > execute once per day are being kicked off again, which is not the
> > > behavior
> > > > I want. I don't have these set to use a CRON based schedule, because
> > > there
> > > > is currently no way of having a CRON scheduled processor run on a
> > single
> > > > node.
> > > >
> > > > --
> > > > Ricky Saltzer
> > > > http://www.cloudera.com
> > > >
> > >
> > >
> > >
> > > --
> > > Sean
> > >
> >
> >
> >
> > --
> > Ricky Saltzer
> > http://www.cloudera.com
> >
>
>
>
> --
> Sean
>
Re: Updating NARs on a running cluster
Posted by Ricky Saltzer <ri...@cloudera.com>.
ahh...the manual process is tedious for sure, but I wouldn't mind doing it
if it actually worked. Unfortunately, if you have processors that execute
once a day, and need to *only *execute once a day, things can get pretty
tricky.
+1 for rolling restarts via the NCM UI.
+1 for schedule state being saved across restarts
On Thu, Mar 26, 2015 at 2:15 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Oh, I was just short-handing your description of your manual process. :)
>
> On Thu, Mar 26, 2015 at 11:10 AM, Ricky Saltzer <ri...@cloudera.com>
> wrote:
>
> > Is there a way to perform a rolling restart? Perhaps undocumented?
> >
> > $ ./bin/nifi.sh
> > Usage nifi {start|stop|run|restart|status|dump|install}
> >
> >
> > On Thu, Mar 26, 2015 at 2:01 PM, Sean Busbey <bu...@cloudera.com>
> wrote:
> >
> > > If the schedule state was properly maintained across a restart, would
> the
> > > rolling-restart-to-update be fine?
> > >
> > > On Thu, Mar 26, 2015 at 10:35 AM, Ricky Saltzer <ri...@cloudera.com>
> > > wrote:
> > >
> > > > Hi -
> > > >
> > > > I have a 3 node NiFi cluster, and periodically I need to update the
> > > custom
> > > > NAR we use for our internal projects. Currently I'm just copying the
> > new
> > > > NAR into the lib directory on each of the nodes (including the
> master),
> > > and
> > > > then restarting each daemon (bin/nifi.sh restart) individually.
> > > >
> > > > I'm curious if there's a better method than the one I'm using. The
> > > biggest
> > > > downside I'm seeing is that all my processors that I have scheduled
> to
> > > > execute once per day are being kicked off again, which is not the
> > > behavior
> > > > I want. I don't have these set to use a CRON based schedule, because
> > > there
> > > > is currently no way of having a CRON scheduled processor run on a
> > single
> > > > node.
> > > >
> > > > --
> > > > Ricky Saltzer
> > > > http://www.cloudera.com
> > > >
> > >
> > >
> > >
> > > --
> > > Sean
> > >
> >
> >
> >
> > --
> > Ricky Saltzer
> > http://www.cloudera.com
> >
>
>
>
> --
> Sean
>
--
Ricky Saltzer
http://www.cloudera.com
Re: Updating NARs on a running cluster
Posted by Sean Busbey <bu...@cloudera.com>.
Oh, I was just short-handing your description of your manual process. :)
On Thu, Mar 26, 2015 at 11:10 AM, Ricky Saltzer <ri...@cloudera.com> wrote:
> Is there a way to perform a rolling restart? Perhaps undocumented?
>
> $ ./bin/nifi.sh
> Usage nifi {start|stop|run|restart|status|dump|install}
>
>
> On Thu, Mar 26, 2015 at 2:01 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > If the schedule state was properly maintained across a restart, would the
> > rolling-restart-to-update be fine?
> >
> > On Thu, Mar 26, 2015 at 10:35 AM, Ricky Saltzer <ri...@cloudera.com>
> > wrote:
> >
> > > Hi -
> > >
> > > I have a 3 node NiFi cluster, and periodically I need to update the
> > custom
> > > NAR we use for our internal projects. Currently I'm just copying the
> new
> > > NAR into the lib directory on each of the nodes (including the master),
> > and
> > > then restarting each daemon (bin/nifi.sh restart) individually.
> > >
> > > I'm curious if there's a better method than the one I'm using. The
> > biggest
> > > downside I'm seeing is that all my processors that I have scheduled to
> > > execute once per day are being kicked off again, which is not the
> > behavior
> > > I want. I don't have these set to use a CRON based schedule, because
> > there
> > > is currently no way of having a CRON scheduled processor run on a
> single
> > > node.
> > >
> > > --
> > > Ricky Saltzer
> > > http://www.cloudera.com
> > >
> >
> >
> >
> > --
> > Sean
> >
>
>
>
> --
> Ricky Saltzer
> http://www.cloudera.com
>
--
Sean
Re: Updating NARs on a running cluster
Posted by Ricky Saltzer <ri...@cloudera.com>.
Is there a way to perform a rolling restart? Perhaps undocumented?
$ ./bin/nifi.sh
Usage nifi {start|stop|run|restart|status|dump|install}
On Thu, Mar 26, 2015 at 2:01 PM, Sean Busbey <bu...@cloudera.com> wrote:
> If the schedule state was properly maintained across a restart, would the
> rolling-restart-to-update be fine?
>
> On Thu, Mar 26, 2015 at 10:35 AM, Ricky Saltzer <ri...@cloudera.com>
> wrote:
>
> > Hi -
> >
> > I have a 3 node NiFi cluster, and periodically I need to update the
> custom
> > NAR we use for our internal projects. Currently I'm just copying the new
> > NAR into the lib directory on each of the nodes (including the master),
> and
> > then restarting each daemon (bin/nifi.sh restart) individually.
> >
> > I'm curious if there's a better method than the one I'm using. The
> biggest
> > downside I'm seeing is that all my processors that I have scheduled to
> > execute once per day are being kicked off again, which is not the
> behavior
> > I want. I don't have these set to use a CRON based schedule, because
> there
> > is currently no way of having a CRON scheduled processor run on a single
> > node.
> >
> > --
> > Ricky Saltzer
> > http://www.cloudera.com
> >
>
>
>
> --
> Sean
>
--
Ricky Saltzer
http://www.cloudera.com
Re: Updating NARs on a running cluster
Posted by Sean Busbey <bu...@cloudera.com>.
If the schedule state was properly maintained across a restart, would the
rolling-restart-to-update be fine?
On Thu, Mar 26, 2015 at 10:35 AM, Ricky Saltzer <ri...@cloudera.com> wrote:
> Hi -
>
> I have a 3 node NiFi cluster, and periodically I need to update the custom
> NAR we use for our internal projects. Currently I'm just copying the new
> NAR into the lib directory on each of the nodes (including the master), and
> then restarting each daemon (bin/nifi.sh restart) individually.
>
> I'm curious if there's a better method than the one I'm using. The biggest
> downside I'm seeing is that all my processors that I have scheduled to
> execute once per day are being kicked off again, which is not the behavior
> I want. I don't have these set to use a CRON based schedule, because there
> is currently no way of having a CRON scheduled processor run on a single
> node.
>
> --
> Ricky Saltzer
> http://www.cloudera.com
>
--
Sean