You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by koustav choudhuri <ko...@gmail.com> on 2016/08/17 22:49:18 UTC
Query related to ExecuteScript
HI All
I am trying to execute 2 different shell scripts ( calling a sqoop job and
calling a hive script) in a sequence. Can I achieve this through NIFI ?
I tried to do it through ExecuteProcess Processor but the issue is Execute
Processor does not accept an input.
Then I moved to ExecuteScript :
Created 2 different python scripts, 1 calling the Sqoop job and the other
calling the Hive script and arranged them in 2 different ExecuteScript
Processors but looks like it doesnt work.
Can you guys tell me if this is achievable in NIFI? Can I call Sqoop/HIVE/
SPARK jobs residing on other servers in a sequence or flow with NIFI?
Thanks
Koustav
Re: Query related to ExecuteScript
Posted by Jagrut Sharma <ja...@gmail.com>.
Koustav - Try checking the permissions for the directory you are trying to
write to. Does the user running nifi have appropriate access?
--
Jagrut
On Wed, Aug 17, 2016 at 4:34 PM, koustav choudhuri <ko...@gmail.com>
wrote:
> Matt
>
> Thanks for Replying , but I am still struggling with ExecuteScript.
>
>
> [image: Inline image 1]
>
>
> in that ExecuteScript all I have is :
>
> Script - set to Python
> Script Body :
> from sys import argv
>
> filename = "nifi_output/nifi_test.txt"
> target = open(filename, 'w')
> target.write("My Name is Bond")
> target.close()
>
> When I run this , Nifi hangs for some reason. Am I doing something grossly
> wrong ?
>
>
>
>
>
>
> On Wed, Aug 17, 2016 at 3:57 PM, Matt Burgess <ma...@gmail.com> wrote:
>
>> If you need an input flowfile, you're probably better off with
>> ExecuteStreamCommand than ExecuteScript for this use case.
>> ExecuteStreamCommand is much like ExecuteProcess but it accepts input flow
>> files.
>>
>> Regards,
>> Matt
>>
>> > On Aug 17, 2016, at 6:49 PM, koustav choudhuri <ko...@gmail.com>
>> wrote:
>> >
>> > HI All
>> >
>> > I am trying to execute 2 different shell scripts ( calling a sqoop job
>> and calling a hive script) in a sequence. Can I achieve this through NIFI ?
>> >
>> > I tried to do it through ExecuteProcess Processor but the issue is
>> Execute Processor does not accept an input.
>> >
>> > Then I moved to ExecuteScript :
>> > Created 2 different python scripts, 1 calling the Sqoop job and the
>> other calling the Hive script and arranged them in 2 different
>> ExecuteScript Processors but looks like it doesnt work.
>> >
>> > Can you guys tell me if this is achievable in NIFI? Can I call
>> Sqoop/HIVE/ SPARK jobs residing on other servers in a sequence or flow with
>> NIFI?
>> >
>> >
>> > Thanks
>> > Koustav
>>
>
Re: Query related to ExecuteScript
Posted by Matt Burgess <ma...@gmail.com>.
Koustav,
Glad to hear it! Let me know if you run into any other issues.
Regards,
Matt
On Thu, Aug 18, 2016 at 6:05 PM, koustav choudhuri <ko...@gmail.com>
wrote:
> I found the issue . Uncommented the parameter java.arg.11 in
> bootstrap.conf which resolved the issue. Now the template runs fine.
>
>
> Thanks
> Koustav
>
> On Thu, Aug 18, 2016 at 1:57 PM, koustav choudhuri <ko...@gmail.com>
> wrote:
>
>> Hi Matt
>>
>> This is what I did :
>>
>> Removed the NIFI directory and killed all the related processes.
>> Installed Nifi 0.7
>> started the service
>> imported your template JythonJsonToJsonExample and started the job
>>
>> This is when the web server stops responding.
>>
>> Errors from bootstrap.log:
>> 2016-08-18 20:49:44,138 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr
>> 2016-08-18 20:49:44,138 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Exception: java.lang.OutOfMemoryError thrown from
>> the UncaughtExceptionHandler in thread "NiFi Web Server-93"
>> 2016-08-18 20:49:44,347 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr
>> 2016-08-18 20:49:44,347 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Exception: java.lang.OutOfMemoryError thrown from
>> the UncaughtExceptionHandler in thread "NiFi Web Server-113"
>> 2016-08-18 20:50:21,388 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr
>> 2016-08-18 20:50:21,388 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Exception: java.lang.OutOfMemoryError thrown from
>> the UncaughtExceptionHandler in thread "NiFi Web Server-118"
>> 2016-08-18 20:50:21,787 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr
>> 2016-08-18 20:50:21,787 ERROR [NiFi logging handler]
>> org.apache.nifi.StdErr Exception: java.lang.OutOfMemoryError thrown from
>> the UncaughtExceptionHandler in thread "NiFi Web Server-120"
>>
>>
>> Errors from nifi-app.log:
>>
>>
>> 2016-08-18 20:49:41,987 WARN [NiFi Web Server-112]
>> o.e.jetty.util.thread.QueuedThreadPool Unexpected thread death:
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3@54b057d5 in NiFi Web
>> Server{STARTED,8<=13<=200,i=3,q=0}
>> 2016-08-18 20:49:41,987 ERROR [NiFi Web Server-112] org.apache.nifi.NiFi
>> An Unknown Error Occurred in Thread Thread[NiFi Web Server-112,5,main]:
>> java.lang.OutOfMemoryError: PermGen space
>> 2016-08-18 20:49:42,759 WARN [NiFi Web Server-111]
>> o.e.jetty.util.thread.QueuedThreadPool Unexpected thread death:
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3@54b057d5 in NiFi Web
>> Server{STARTED,8<=13<=200,i=4,q=0}
>> 2016-08-18 20:49:42,759 ERROR [NiFi Web Server-111] org.apache.nifi.NiFi
>> An Unknown Error Occurred in Thread Thread[NiFi Web Server-111,5,main]:
>> java.lang.OutOfMemoryError: PermGen space
>>
>>
>> free -m results in :
>>
>> [image: Inline image 1]
>>
>> Any pointers on what is going wrong ?
>>
>>
>> Thanks
>> Koustav
>>
>>
>>
>> On Wed, Aug 17, 2016 at 7:08 PM, Matt Burgess <ma...@gmail.com>
>> wrote:
>>
>>> Koustav,
>>>
>>> I took your script (with GenerateFlowFile before it) and ran just fine
>>> (meaning the file is created and the processor does not hang). When I
>>> didn't have write permissions for the target folder, I got an
>>> error/bulletin telling me this. When I did have write permissions, it works
>>> fine.
>>>
>>> However I should mention that if you want something to go to the PutFile
>>> (in your flow above), you will not get anything output (to the flow) with
>>> your current script. Instead you need to create/reuse a flow file and then
>>> use the session to transfer it. There is an example of how to do this on my
>>> blog: http://funnifi.blogspot.com/2016/03/executescript-json
>>> -to-json-revisited_14.html
>>>
>>> Regards,
>>> Matt
>>>
>>> On Wed, Aug 17, 2016 at 7:34 PM, koustav choudhuri <
>>> koustav1982@gmail.com> wrote:
>>>
>>>> Matt
>>>>
>>>> Thanks for Replying , but I am still struggling with ExecuteScript.
>>>>
>>>>
>>>> [image: Inline image 1]
>>>>
>>>>
>>>> in that ExecuteScript all I have is :
>>>>
>>>> Script - set to Python
>>>> Script Body :
>>>> from sys import argv
>>>>
>>>> filename = "nifi_output/nifi_test.txt"
>>>> target = open(filename, 'w')
>>>> target.write("My Name is Bond")
>>>> target.close()
>>>>
>>>> When I run this , Nifi hangs for some reason. Am I doing something
>>>> grossly wrong ?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Aug 17, 2016 at 3:57 PM, Matt Burgess <ma...@gmail.com>
>>>> wrote:
>>>>
>>>>> If you need an input flowfile, you're probably better off with
>>>>> ExecuteStreamCommand than ExecuteScript for this use case.
>>>>> ExecuteStreamCommand is much like ExecuteProcess but it accepts input flow
>>>>> files.
>>>>>
>>>>> Regards,
>>>>> Matt
>>>>>
>>>>> > On Aug 17, 2016, at 6:49 PM, koustav choudhuri <
>>>>> koustav1982@gmail.com> wrote:
>>>>> >
>>>>> > HI All
>>>>> >
>>>>> > I am trying to execute 2 different shell scripts ( calling a sqoop
>>>>> job and calling a hive script) in a sequence. Can I achieve this through
>>>>> NIFI ?
>>>>> >
>>>>> > I tried to do it through ExecuteProcess Processor but the issue is
>>>>> Execute Processor does not accept an input.
>>>>> >
>>>>> > Then I moved to ExecuteScript :
>>>>> > Created 2 different python scripts, 1 calling the Sqoop job and the
>>>>> other calling the Hive script and arranged them in 2 different
>>>>> ExecuteScript Processors but looks like it doesnt work.
>>>>> >
>>>>> > Can you guys tell me if this is achievable in NIFI? Can I call
>>>>> Sqoop/HIVE/ SPARK jobs residing on other servers in a sequence or flow with
>>>>> NIFI?
>>>>> >
>>>>> >
>>>>> > Thanks
>>>>> > Koustav
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Query related to ExecuteScript
Posted by koustav choudhuri <ko...@gmail.com>.
I found the issue . Uncommented the parameter java.arg.11 in bootstrap.conf
which resolved the issue. Now the template runs fine.
Thanks
Koustav
On Thu, Aug 18, 2016 at 1:57 PM, koustav choudhuri <ko...@gmail.com>
wrote:
> Hi Matt
>
> This is what I did :
>
> Removed the NIFI directory and killed all the related processes.
> Installed Nifi 0.7
> started the service
> imported your template JythonJsonToJsonExample and started the job
>
> This is when the web server stops responding.
>
> Errors from bootstrap.log:
> 2016-08-18 20:49:44,138 ERROR [NiFi logging handler] org.apache.nifi.StdErr
> 2016-08-18 20:49:44,138 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Exception: java.lang.OutOfMemoryError thrown from
> the UncaughtExceptionHandler in thread "NiFi Web Server-93"
> 2016-08-18 20:49:44,347 ERROR [NiFi logging handler] org.apache.nifi.StdErr
> 2016-08-18 20:49:44,347 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Exception: java.lang.OutOfMemoryError thrown from
> the UncaughtExceptionHandler in thread "NiFi Web Server-113"
> 2016-08-18 20:50:21,388 ERROR [NiFi logging handler] org.apache.nifi.StdErr
> 2016-08-18 20:50:21,388 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Exception: java.lang.OutOfMemoryError thrown from
> the UncaughtExceptionHandler in thread "NiFi Web Server-118"
> 2016-08-18 20:50:21,787 ERROR [NiFi logging handler] org.apache.nifi.StdErr
> 2016-08-18 20:50:21,787 ERROR [NiFi logging handler]
> org.apache.nifi.StdErr Exception: java.lang.OutOfMemoryError thrown from
> the UncaughtExceptionHandler in thread "NiFi Web Server-120"
>
>
> Errors from nifi-app.log:
>
>
> 2016-08-18 20:49:41,987 WARN [NiFi Web Server-112] o.e.jetty.util.thread.QueuedThreadPool
> Unexpected thread death: org.eclipse.jetty.util.thread.
> QueuedThreadPool$3@54b057d5 in NiFi Web Server{STARTED,8<=13<=200,i=3,q=0}
> 2016-08-18 20:49:41,987 ERROR [NiFi Web Server-112] org.apache.nifi.NiFi
> An Unknown Error Occurred in Thread Thread[NiFi Web Server-112,5,main]:
> java.lang.OutOfMemoryError: PermGen space
> 2016-08-18 20:49:42,759 WARN [NiFi Web Server-111] o.e.jetty.util.thread.QueuedThreadPool
> Unexpected thread death: org.eclipse.jetty.util.thread.
> QueuedThreadPool$3@54b057d5 in NiFi Web Server{STARTED,8<=13<=200,i=4,q=0}
> 2016-08-18 20:49:42,759 ERROR [NiFi Web Server-111] org.apache.nifi.NiFi
> An Unknown Error Occurred in Thread Thread[NiFi Web Server-111,5,main]:
> java.lang.OutOfMemoryError: PermGen space
>
>
> free -m results in :
>
> [image: Inline image 1]
>
> Any pointers on what is going wrong ?
>
>
> Thanks
> Koustav
>
>
>
> On Wed, Aug 17, 2016 at 7:08 PM, Matt Burgess <ma...@gmail.com> wrote:
>
>> Koustav,
>>
>> I took your script (with GenerateFlowFile before it) and ran just fine
>> (meaning the file is created and the processor does not hang). When I
>> didn't have write permissions for the target folder, I got an
>> error/bulletin telling me this. When I did have write permissions, it works
>> fine.
>>
>> However I should mention that if you want something to go to the PutFile
>> (in your flow above), you will not get anything output (to the flow) with
>> your current script. Instead you need to create/reuse a flow file and then
>> use the session to transfer it. There is an example of how to do this on my
>> blog: http://funnifi.blogspot.com/2016/03/executescript-json
>> -to-json-revisited_14.html
>>
>> Regards,
>> Matt
>>
>> On Wed, Aug 17, 2016 at 7:34 PM, koustav choudhuri <koustav1982@gmail.com
>> > wrote:
>>
>>> Matt
>>>
>>> Thanks for Replying , but I am still struggling with ExecuteScript.
>>>
>>>
>>> [image: Inline image 1]
>>>
>>>
>>> in that ExecuteScript all I have is :
>>>
>>> Script - set to Python
>>> Script Body :
>>> from sys import argv
>>>
>>> filename = "nifi_output/nifi_test.txt"
>>> target = open(filename, 'w')
>>> target.write("My Name is Bond")
>>> target.close()
>>>
>>> When I run this , Nifi hangs for some reason. Am I doing something
>>> grossly wrong ?
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Aug 17, 2016 at 3:57 PM, Matt Burgess <ma...@gmail.com>
>>> wrote:
>>>
>>>> If you need an input flowfile, you're probably better off with
>>>> ExecuteStreamCommand than ExecuteScript for this use case.
>>>> ExecuteStreamCommand is much like ExecuteProcess but it accepts input flow
>>>> files.
>>>>
>>>> Regards,
>>>> Matt
>>>>
>>>> > On Aug 17, 2016, at 6:49 PM, koustav choudhuri <ko...@gmail.com>
>>>> wrote:
>>>> >
>>>> > HI All
>>>> >
>>>> > I am trying to execute 2 different shell scripts ( calling a sqoop
>>>> job and calling a hive script) in a sequence. Can I achieve this through
>>>> NIFI ?
>>>> >
>>>> > I tried to do it through ExecuteProcess Processor but the issue is
>>>> Execute Processor does not accept an input.
>>>> >
>>>> > Then I moved to ExecuteScript :
>>>> > Created 2 different python scripts, 1 calling the Sqoop job and the
>>>> other calling the Hive script and arranged them in 2 different
>>>> ExecuteScript Processors but looks like it doesnt work.
>>>> >
>>>> > Can you guys tell me if this is achievable in NIFI? Can I call
>>>> Sqoop/HIVE/ SPARK jobs residing on other servers in a sequence or flow with
>>>> NIFI?
>>>> >
>>>> >
>>>> > Thanks
>>>> > Koustav
>>>>
>>>
>>>
>>
>
Re: Query related to ExecuteScript
Posted by koustav choudhuri <ko...@gmail.com>.
Hi Matt
This is what I did :
Removed the NIFI directory and killed all the related processes.
Installed Nifi 0.7
started the service
imported your template JythonJsonToJsonExample and started the job
This is when the web server stops responding.
Errors from bootstrap.log:
2016-08-18 20:49:44,138 ERROR [NiFi logging handler] org.apache.nifi.StdErr
2016-08-18 20:49:44,138 ERROR [NiFi logging handler] org.apache.nifi.StdErr
Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "NiFi Web Server-93"
2016-08-18 20:49:44,347 ERROR [NiFi logging handler] org.apache.nifi.StdErr
2016-08-18 20:49:44,347 ERROR [NiFi logging handler] org.apache.nifi.StdErr
Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "NiFi Web Server-113"
2016-08-18 20:50:21,388 ERROR [NiFi logging handler] org.apache.nifi.StdErr
2016-08-18 20:50:21,388 ERROR [NiFi logging handler] org.apache.nifi.StdErr
Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "NiFi Web Server-118"
2016-08-18 20:50:21,787 ERROR [NiFi logging handler] org.apache.nifi.StdErr
2016-08-18 20:50:21,787 ERROR [NiFi logging handler] org.apache.nifi.StdErr
Exception: java.lang.OutOfMemoryError thrown from the
UncaughtExceptionHandler in thread "NiFi Web Server-120"
Errors from nifi-app.log:
2016-08-18 20:49:41,987 WARN [NiFi Web Server-112]
o.e.jetty.util.thread.QueuedThreadPool Unexpected thread death:
org.eclipse.jetty.util.thread.QueuedThreadPool$3@54b057d5 in NiFi Web
Server{STARTED,8<=13<=200,i=3,q=0}
2016-08-18 20:49:41,987 ERROR [NiFi Web Server-112] org.apache.nifi.NiFi An
Unknown Error Occurred in Thread Thread[NiFi Web Server-112,5,main]:
java.lang.OutOfMemoryError: PermGen space
2016-08-18 20:49:42,759 WARN [NiFi Web Server-111]
o.e.jetty.util.thread.QueuedThreadPool Unexpected thread death:
org.eclipse.jetty.util.thread.QueuedThreadPool$3@54b057d5 in NiFi Web
Server{STARTED,8<=13<=200,i=4,q=0}
2016-08-18 20:49:42,759 ERROR [NiFi Web Server-111] org.apache.nifi.NiFi An
Unknown Error Occurred in Thread Thread[NiFi Web Server-111,5,main]:
java.lang.OutOfMemoryError: PermGen space
free -m results in :
[image: Inline image 1]
Any pointers on what is going wrong ?
Thanks
Koustav
On Wed, Aug 17, 2016 at 7:08 PM, Matt Burgess <ma...@gmail.com> wrote:
> Koustav,
>
> I took your script (with GenerateFlowFile before it) and ran just fine
> (meaning the file is created and the processor does not hang). When I
> didn't have write permissions for the target folder, I got an
> error/bulletin telling me this. When I did have write permissions, it works
> fine.
>
> However I should mention that if you want something to go to the PutFile
> (in your flow above), you will not get anything output (to the flow) with
> your current script. Instead you need to create/reuse a flow file and then
> use the session to transfer it. There is an example of how to do this on my
> blog: http://funnifi.blogspot.com/2016/03/executescript-
> json-to-json-revisited_14.html
>
> Regards,
> Matt
>
> On Wed, Aug 17, 2016 at 7:34 PM, koustav choudhuri <ko...@gmail.com>
> wrote:
>
>> Matt
>>
>> Thanks for Replying , but I am still struggling with ExecuteScript.
>>
>>
>> [image: Inline image 1]
>>
>>
>> in that ExecuteScript all I have is :
>>
>> Script - set to Python
>> Script Body :
>> from sys import argv
>>
>> filename = "nifi_output/nifi_test.txt"
>> target = open(filename, 'w')
>> target.write("My Name is Bond")
>> target.close()
>>
>> When I run this , Nifi hangs for some reason. Am I doing something
>> grossly wrong ?
>>
>>
>>
>>
>>
>>
>> On Wed, Aug 17, 2016 at 3:57 PM, Matt Burgess <ma...@gmail.com>
>> wrote:
>>
>>> If you need an input flowfile, you're probably better off with
>>> ExecuteStreamCommand than ExecuteScript for this use case.
>>> ExecuteStreamCommand is much like ExecuteProcess but it accepts input flow
>>> files.
>>>
>>> Regards,
>>> Matt
>>>
>>> > On Aug 17, 2016, at 6:49 PM, koustav choudhuri <ko...@gmail.com>
>>> wrote:
>>> >
>>> > HI All
>>> >
>>> > I am trying to execute 2 different shell scripts ( calling a sqoop job
>>> and calling a hive script) in a sequence. Can I achieve this through NIFI ?
>>> >
>>> > I tried to do it through ExecuteProcess Processor but the issue is
>>> Execute Processor does not accept an input.
>>> >
>>> > Then I moved to ExecuteScript :
>>> > Created 2 different python scripts, 1 calling the Sqoop job and the
>>> other calling the Hive script and arranged them in 2 different
>>> ExecuteScript Processors but looks like it doesnt work.
>>> >
>>> > Can you guys tell me if this is achievable in NIFI? Can I call
>>> Sqoop/HIVE/ SPARK jobs residing on other servers in a sequence or flow with
>>> NIFI?
>>> >
>>> >
>>> > Thanks
>>> > Koustav
>>>
>>
>>
>
Re: Query related to ExecuteScript
Posted by Matt Burgess <ma...@gmail.com>.
Koustav,
I took your script (with GenerateFlowFile before it) and ran just fine
(meaning the file is created and the processor does not hang). When I
didn't have write permissions for the target folder, I got an
error/bulletin telling me this. When I did have write permissions, it works
fine.
However I should mention that if you want something to go to the PutFile
(in your flow above), you will not get anything output (to the flow) with
your current script. Instead you need to create/reuse a flow file and then
use the session to transfer it. There is an example of how to do this on my
blog:
http://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html
Regards,
Matt
On Wed, Aug 17, 2016 at 7:34 PM, koustav choudhuri <ko...@gmail.com>
wrote:
> Matt
>
> Thanks for Replying , but I am still struggling with ExecuteScript.
>
>
> [image: Inline image 1]
>
>
> in that ExecuteScript all I have is :
>
> Script - set to Python
> Script Body :
> from sys import argv
>
> filename = "nifi_output/nifi_test.txt"
> target = open(filename, 'w')
> target.write("My Name is Bond")
> target.close()
>
> When I run this , Nifi hangs for some reason. Am I doing something grossly
> wrong ?
>
>
>
>
>
>
> On Wed, Aug 17, 2016 at 3:57 PM, Matt Burgess <ma...@gmail.com> wrote:
>
>> If you need an input flowfile, you're probably better off with
>> ExecuteStreamCommand than ExecuteScript for this use case.
>> ExecuteStreamCommand is much like ExecuteProcess but it accepts input flow
>> files.
>>
>> Regards,
>> Matt
>>
>> > On Aug 17, 2016, at 6:49 PM, koustav choudhuri <ko...@gmail.com>
>> wrote:
>> >
>> > HI All
>> >
>> > I am trying to execute 2 different shell scripts ( calling a sqoop job
>> and calling a hive script) in a sequence. Can I achieve this through NIFI ?
>> >
>> > I tried to do it through ExecuteProcess Processor but the issue is
>> Execute Processor does not accept an input.
>> >
>> > Then I moved to ExecuteScript :
>> > Created 2 different python scripts, 1 calling the Sqoop job and the
>> other calling the Hive script and arranged them in 2 different
>> ExecuteScript Processors but looks like it doesnt work.
>> >
>> > Can you guys tell me if this is achievable in NIFI? Can I call
>> Sqoop/HIVE/ SPARK jobs residing on other servers in a sequence or flow with
>> NIFI?
>> >
>> >
>> > Thanks
>> > Koustav
>>
>
>
Re: Query related to ExecuteScript
Posted by koustav choudhuri <ko...@gmail.com>.
Matt
Thanks for Replying , but I am still struggling with ExecuteScript.
[image: Inline image 1]
in that ExecuteScript all I have is :
Script - set to Python
Script Body :
from sys import argv
filename = "nifi_output/nifi_test.txt"
target = open(filename, 'w')
target.write("My Name is Bond")
target.close()
When I run this , Nifi hangs for some reason. Am I doing something grossly
wrong ?
On Wed, Aug 17, 2016 at 3:57 PM, Matt Burgess <ma...@gmail.com> wrote:
> If you need an input flowfile, you're probably better off with
> ExecuteStreamCommand than ExecuteScript for this use case.
> ExecuteStreamCommand is much like ExecuteProcess but it accepts input flow
> files.
>
> Regards,
> Matt
>
> > On Aug 17, 2016, at 6:49 PM, koustav choudhuri <ko...@gmail.com>
> wrote:
> >
> > HI All
> >
> > I am trying to execute 2 different shell scripts ( calling a sqoop job
> and calling a hive script) in a sequence. Can I achieve this through NIFI ?
> >
> > I tried to do it through ExecuteProcess Processor but the issue is
> Execute Processor does not accept an input.
> >
> > Then I moved to ExecuteScript :
> > Created 2 different python scripts, 1 calling the Sqoop job and the
> other calling the Hive script and arranged them in 2 different
> ExecuteScript Processors but looks like it doesnt work.
> >
> > Can you guys tell me if this is achievable in NIFI? Can I call
> Sqoop/HIVE/ SPARK jobs residing on other servers in a sequence or flow with
> NIFI?
> >
> >
> > Thanks
> > Koustav
>
Re: Query related to ExecuteScript
Posted by Matt Burgess <ma...@gmail.com>.
If you need an input flowfile, you're probably better off with ExecuteStreamCommand than ExecuteScript for this use case. ExecuteStreamCommand is much like ExecuteProcess but it accepts input flow files.
Regards,
Matt
> On Aug 17, 2016, at 6:49 PM, koustav choudhuri <ko...@gmail.com> wrote:
>
> HI All
>
> I am trying to execute 2 different shell scripts ( calling a sqoop job and calling a hive script) in a sequence. Can I achieve this through NIFI ?
>
> I tried to do it through ExecuteProcess Processor but the issue is Execute Processor does not accept an input.
>
> Then I moved to ExecuteScript :
> Created 2 different python scripts, 1 calling the Sqoop job and the other calling the Hive script and arranged them in 2 different ExecuteScript Processors but looks like it doesnt work.
>
> Can you guys tell me if this is achievable in NIFI? Can I call Sqoop/HIVE/ SPARK jobs residing on other servers in a sequence or flow with NIFI?
>
>
> Thanks
> Koustav