ExecuteScript and flowfiles

[DAVID SMITH <da...@btinternet.com>]

Hi

I have a question from a colleague who is using nifi 0.5.1, he has some files coming in which he needs to break up into 3 constituent parts.
He has a python script which should do this but what he wants to know is can he either

1)   send all three parts down one relationship as three separate flowfiles, from his python script, or
2) can he create three relationships from within his python script, to send a flowfile down each?

Does anyone have any examples of doing either option or can advise how to do this.

Many thanks
Dave

Sent from Yahoo! Mail on Android

Re: ExecuteScript and flowfiles

[Matt Burgess <ma...@gmail.com>]

Dave,

Both options are possible, though note the scripting processors use
Jython not Python, so hopefully the scripts would work but might need
a little refactor.

The first option is probably best solved with the ExecuteScript
processor, the second is probably best solved with
InvokeScriptedProcessor. I have examples of similar use cases and of
using Jython on my blog:

http://funnifi.blogspot.com/

Regards,
Matt

On Mon, Jul 18, 2016 at 12:30 PM, DAVID SMITH
<da...@btinternet.com> wrote:
> Hi
>
> I have a question from a colleague who is using nifi 0.5.1, he has some files coming in which he needs to break up into 3 constituent parts.
> He has a python script which should do this but what he wants to know is can he either
>
> 1)   send all three parts down one relationship as three separate flowfiles, from his python script, or
> 2) can he create three relationships from within his python script, to send a flowfile down each?
>
> Does anyone have any examples of doing either option or can advise how to do this.
>
> Many thanks
> Dave
>
> Sent from Yahoo! Mail on Android
>

Re: ExecuteScript and flowfiles

[Andy LoPresto <al...@apache.org>]

Dave,

Either option is plausible, but the first is probably preferable. I would recommend the Python script should add an attribute to each flowfile with some flag or indicator (i.e. which virtual relationship it would follow) and have a single connection out of the ExecuteScript processor for success connected to a RouteOnAttribute processor. This processor is designed to have multiple outgoing connections and will route the incoming flowfiles based on attributes, as the name would indicate.

Doing this the second way would require additional coding in the Python script to reference new relationships, and this can be tricky.

Matt Burgess has some good examples of using Python scripts in the ExecuteScript processor on his blog [1].

[1] https://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html <https://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html>


Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jul 18, 2016, at 9:30 AM, DAVID SMITH <da...@btinternet.com> wrote:
> 
> Hi
> 
> I have a question from a colleague who is using nifi 0.5.1, he has some files coming in which he needs to break up into 3 constituent parts.
> He has a python script which should do this but what he wants to know is can he either
> 
> 1)   send all three parts down one relationship as three separate flowfiles, from his python script, or
> 2) can he create three relationships from within his python script, to send a flowfile down each?
> 
> Does anyone have any examples of doing either option or can advise how to do this.
> 
> Many thanks
> Dave
> 
> Sent from Yahoo! Mail on Android
>