You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Dirk Arends <di...@fontis.com.au> on 2021/07/02 01:29:40 UTC
Invalidating InvokeScriptedProcessors when Script Files changes
Hi All,
I have a query regarding InvokeScriptedProcessors and the best way to
invalidate a script context. I'm using a number of InvokeScriptedProcessors
with a “Script File” set. Each time I update the script at that path,
restarting the InvokeScriptedProcessor does not cause the updated script to
be re-evaluated.
From reading the InvokeScriptedProcessor implementation I can see that the
script will be re-evaluated when one of the Script Engine, Script File
(path), Script Body, or Module Directory properties are modified. This is
sufficient when using “Script Body”, but in the case of a Script File the
properties don’t change and therefore the script is not re-evaluated.
My current workaround is to:
1. Stop the processor
2. Edit the Script Body and add a character
3. Apply changes
4. Edit the Script Body and remove the character
5. Apply changes
6. Start the processor again
* I could use any of the 4 properties but Script Body has been the easiest
since it's empty.
This obviously isn't ideal and isn't something I'm keen to integrate in a
deployment process (where my current thinking is to make NiFi API requests
to mimic the process).
Is anyone able to suggest a better solution to this problem?
Regards,
--
Dirk Arends
Re: Invalidating InvokeScriptedProcessors when Script Files changes
Posted by Dirk Arends <di...@fontis.com.au>.
Hi Matt,
I agree that re-evaluating on the fly would not be desirable. Could a file
watcher could be hooked into the existing `AtomicBoolean scriptNeedsReload`
allowing the new script to be evaluated on start? Potentially this
functionality could be behind a new property "Re-evaluate script on start"
which defaults to false. This way existing functionality will remain and
does not introduce a breaking change.
I had expected restarting the processor would still be a part of my
deployment process, while avoiding programmatically invalidating the script
context.
Regards,
Dirk Arends
On Wed, 7 Jul 2021 at 08:29, Matt Burgess <ma...@apache.org> wrote:
> Dirk,
>
> We could look at adding a FileWatcher or something to
> InvokeScriptedProcessor, but I doubt we'd want to allow re-evaluating
> the script on the fly, maybe we would just set a flag indicating a
> change was detected and the next time the processor is started or the
> script would be evaluated, we'd reload the file contents at that
> point.
>
> IIRC, using Script File was originally intended for "finished" scripts
> to be located on each node of a cluster (or shared), so the sysadmins
> could control the file rather than the flow designers. If you're
> updating/testing scripts, I recommend copying the contents to Script
> Body until it works the way you want, then copying the script back to
> the file.
>
> Regards,
> Matt
>
> On Thu, Jul 1, 2021 at 9:30 PM Dirk Arends <di...@fontis.com.au>
> wrote:
> >
> > Hi All,
> >
> > I have a query regarding InvokeScriptedProcessors and the best way to
> invalidate a script context. I'm using a number of InvokeScriptedProcessors
> with a “Script File” set. Each time I update the script at that path,
> restarting the InvokeScriptedProcessor does not cause the updated script to
> be re-evaluated.
> >
> > From reading the InvokeScriptedProcessor implementation I can see that
> the script will be re-evaluated when one of the Script Engine, Script File
> (path), Script Body, or Module Directory properties are modified. This is
> sufficient when using “Script Body”, but in the case of a Script File the
> properties don’t change and therefore the script is not re-evaluated.
> >
> > My current workaround is to:
> >
> > 1. Stop the processor
> > 2. Edit the Script Body and add a character
> > 3. Apply changes
> > 4. Edit the Script Body and remove the character
> > 5. Apply changes
> > 6. Start the processor again
> > * I could use any of the 4 properties but Script Body has been the
> easiest since it's empty.
> >
> > This obviously isn't ideal and isn't something I'm keen to integrate in
> a deployment process (where my current thinking is to make NiFi API
> requests to mimic the process).
> >
> > Is anyone able to suggest a better solution to this problem?
> >
> > Regards,
> >
> > --
> > Dirk Arends
>
--
Regards,
--
Dirk Arends
Re: Invalidating InvokeScriptedProcessors when Script Files changes
Posted by Matt Burgess <ma...@apache.org>.
Dirk,
We could look at adding a FileWatcher or something to
InvokeScriptedProcessor, but I doubt we'd want to allow re-evaluating
the script on the fly, maybe we would just set a flag indicating a
change was detected and the next time the processor is started or the
script would be evaluated, we'd reload the file contents at that
point.
IIRC, using Script File was originally intended for "finished" scripts
to be located on each node of a cluster (or shared), so the sysadmins
could control the file rather than the flow designers. If you're
updating/testing scripts, I recommend copying the contents to Script
Body until it works the way you want, then copying the script back to
the file.
Regards,
Matt
On Thu, Jul 1, 2021 at 9:30 PM Dirk Arends <di...@fontis.com.au> wrote:
>
> Hi All,
>
> I have a query regarding InvokeScriptedProcessors and the best way to invalidate a script context. I'm using a number of InvokeScriptedProcessors with a “Script File” set. Each time I update the script at that path, restarting the InvokeScriptedProcessor does not cause the updated script to be re-evaluated.
>
> From reading the InvokeScriptedProcessor implementation I can see that the script will be re-evaluated when one of the Script Engine, Script File (path), Script Body, or Module Directory properties are modified. This is sufficient when using “Script Body”, but in the case of a Script File the properties don’t change and therefore the script is not re-evaluated.
>
> My current workaround is to:
>
> 1. Stop the processor
> 2. Edit the Script Body and add a character
> 3. Apply changes
> 4. Edit the Script Body and remove the character
> 5. Apply changes
> 6. Start the processor again
> * I could use any of the 4 properties but Script Body has been the easiest since it's empty.
>
> This obviously isn't ideal and isn't something I'm keen to integrate in a deployment process (where my current thinking is to make NiFi API requests to mimic the process).
>
> Is anyone able to suggest a better solution to this problem?
>
> Regards,
>
> --
> Dirk Arends