You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@airavata.apache.org by Heejoon Chae <he...@cs.indiana.edu> on 2013/06/05 04:06:53 UTC
Question for Airavata For-Each component
Hi,
I have quick question about Airavata For-Each component.
It seems that ForEach component takes String array as an input and pass
them to next component one by one.
My question is how to create string array in shell script to pass it to
ForEach component?
I normally generate string result like
echo "output=xxx"
at the end of script and connect it as an input of next component in
Airavata workflow to pass the result and let next component know the input
is arrived.
Thanks,
Heejoon.
Re: Question for Airavata For-Each component
Posted by 채희준 <ic...@gmail.com>.
Thank you!
Heejoon.
2013. 6. 5. 오후 11:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
> Following script will be useful for you..
>
> echo $@
> echo $*
> echo $#
> for arg; do
> printf '%s\n' "$arg"
> done
>
>
> Lahir
>
>
> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>> Hi Lahiru,
>>
>> Thank you for your answer.
>>
>> By the way, how to create string array in shell script, so that I can pass it to ForEach component?
>>
>> Thank you,
>> Heejoon.
>>
>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>>
>>> Hi Heejoon,
>>>
>>> We take string array as input but we invoke the script with each element of the string array multiple times, in Foreach we don't pass whole string array to the application script at once. So you really don't have to handle a string array in your script.
>>>
>>> Lahiru
>>>
>>>
>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>>>> Hi,
>>>>
>>>> I have quick question about Airavata For-Each component.
>>>>
>>>> It seems that ForEach component takes String array as an input and pass them to next component one by one.
>>>>
>>>> My question is how to create string array in shell script to pass it to ForEach component?
>>>>
>>>> I normally generate string result like
>>>>
>>>> echo "output=xxx"
>>>>
>>>> at the end of script and connect it as an input of next component in Airavata workflow to pass the result and let next component know the input is arrived.
>>>>
>>>> Thanks,
>>>> Heejoon.
>>>
>>>
>>>
>>> --
>>> System Analyst Programmer
>>> PTI Lab
>>> Indiana University
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
Re: Question for Airavata For-Each component
Posted by Heejoon Chae <he...@cs.indiana.edu>.
Thank you for taking your time for this.
I have also tested multiple ways with generating xml structure as
suggested, but still it doesn't work.
Since output should looks like 'name=value' to be recognized as output,
I tried like,
***with setting the output type as String type in my script component
1. echo "output=<ComponentOneOrWhateverOutputParamname><val>a</val>..."
=> just recognized as string and Xbaya say 'No array values found for
ForEach component"
2. echo "output=<ComponentOneOrWhateverOutputParamname><vals>a</vals>..."
=> just recognized as string and Xbaya say 'No array values found for
ForEach component'
3. echo "output=<ComponentOneOrWhateverOutputParamname><value>a</value>..."
=> just recognized as string and Xbaya say 'No array values found for
ForEach component"
4. echo "<ComponentOneOrWhateverOutputParamname><val>a</val>..."
=> Fail to retrieve result. no "output=.." found.
***with setting the output type as StringArray type in my script component
with all above testing,
=>output=null, 'No array values found for foreach ...'
Thanks,
Heejoon.
On Wed, Jun 12, 2013 at 12:22 PM, Chathura Herath <ch...@cs.indiana.edu>wrote:
> Will you be able to try generating an output from your service with
> the following xml structure and check.
>
> <ComponentOneOrWhateverOutputParamname>
> <val>a</val>
> <val>b</val>
>
> ..
> </ComponentOneOrWhateverOutputParamname>
>
> I remember XBaya Foreach was able to unwind that above array with
> element names "val" or "vals" and run the foreachs in parallel.
>
> Can someone verify this. If not i ll have a look this Saturday.
>
>
>
> On Tue, Jun 11, 2013 at 9:26 AM, Suresh Marru <sm...@apache.org> wrote:
> > Hi Heejoon,
> >
> > Normally we use for-each on files, so the way to handle that is let all
> the
> > output files go into a outputData directory and iterate over them. But I
> see
> > what you are doing here. Let me try this and get back to you within the
> next
> > day.
> >
> > Thanks for your patience,
> > Suresh
> >
> > On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> >
> > Hi, Airavata users,
> >
> > Do you have any advises or experiences for my previous mail about ForEach
> > component?
> > I am still struggling with generating input for ForEach component in my
> > script :/
> > Any comment will be helpful to me to use the component :)
> >
> > Thank you!
> >
> > ---------- Forwarded message ----------
> > From: Heejoon Chae <he...@cs.indiana.edu>
> > Date: Fri, Jun 7, 2013 at 5:44 PM
> > Subject: Re: Question for Airavata For-Each component
> > To: "users@airavata.apache.org" <us...@airavata.apache.org>
> >
> >
> > Hi, Lahir,
> >
> > I succeeded to use ForEach component with attaching 'Input' component in
> > front of ForEach componet with comma separated value(i.e "a,b,c") as
> input
> > like in attached figure. ForEach component works well with Input
> component
> > with comma separate value.
> >
> > Now I want to mimic what Input component does within my script as in
> another
> > figure. This is required to run ForEach on generated data not on data
> > inputed from user.
> >
> > I tired several ways to generate a input for ForEach component in my
> script
> > including what you suggested in previous mail, but I couldn't. Please
> help
> > me to solve this problem.
> >
> > Thank you,
> > Heejoon.
> >
> >
> > On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>
> > wrote:
> >>
> >> Following script will be useful for you..
> >>
> >> echo $@
> >> echo $*
> >> echo $#
> >> for arg; do
> >> printf '%s\n' "$arg"
> >> done
> >>
> >>
> >> Lahir
> >>
> >>
> >> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
> >>>
> >>> Hi Lahiru,
> >>>
> >>> Thank you for your answer.
> >>>
> >>> By the way, how to create string array in shell script, so that I can
> >>> pass it to ForEach component?
> >>>
> >>> Thank you,
> >>> Heejoon.
> >>>
> >>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
> >>>
> >>> Hi Heejoon,
> >>>
> >>> We take string array as input but we invoke the script with each
> element
> >>> of the string array multiple times, in Foreach we don't pass whole
> string
> >>> array to the application script at once. So you really don't have to
> handle
> >>> a string array in your script.
> >>>
> >>> Lahiru
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>
> >>> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> I have quick question about Airavata For-Each component.
> >>>>
> >>>> It seems that ForEach component takes String array as an input and
> pass
> >>>> them to next component one by one.
> >>>>
> >>>> My question is how to create string array in shell script to pass it
> to
> >>>> ForEach component?
> >>>>
> >>>> I normally generate string result like
> >>>>
> >>>> echo "output=xxx"
> >>>>
> >>>> at the end of script and connect it as an input of next component in
> >>>> Airavata workflow to pass the result and let next component know the
> input
> >>>> is arrived.
> >>>>
> >>>> Thanks,
> >>>> Heejoon.
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> System Analyst Programmer
> >>> PTI Lab
> >>> Indiana University
> >>
> >>
> >>
> >>
> >> --
> >> System Analyst Programmer
> >> PTI Lab
> >> Indiana University
> >
> >
> >
> > <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
> 3.11.31
> > PM.png>
> >
> >
>
>
>
> --
> Chathura Herath Ph.D
> http://people.apache.org/~chathura/
> http://chathurah.blogspot.com/
>
Re: Question for Airavata For-Each component
Posted by Chathura Herath <ch...@cs.indiana.edu>.
Will you be able to try generating an output from your service with
the following xml structure and check.
<ComponentOneOrWhateverOutputParamname>
<val>a</val>
<val>b</val>
..
</ComponentOneOrWhateverOutputParamname>
I remember XBaya Foreach was able to unwind that above array with
element names "val" or "vals" and run the foreachs in parallel.
Can someone verify this. If not i ll have a look this Saturday.
On Tue, Jun 11, 2013 at 9:26 AM, Suresh Marru <sm...@apache.org> wrote:
> Hi Heejoon,
>
> Normally we use for-each on files, so the way to handle that is let all the
> output files go into a outputData directory and iterate over them. But I see
> what you are doing here. Let me try this and get back to you within the next
> day.
>
> Thanks for your patience,
> Suresh
>
> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>
> Hi, Airavata users,
>
> Do you have any advises or experiences for my previous mail about ForEach
> component?
> I am still struggling with generating input for ForEach component in my
> script :/
> Any comment will be helpful to me to use the component :)
>
> Thank you!
>
> ---------- Forwarded message ----------
> From: Heejoon Chae <he...@cs.indiana.edu>
> Date: Fri, Jun 7, 2013 at 5:44 PM
> Subject: Re: Question for Airavata For-Each component
> To: "users@airavata.apache.org" <us...@airavata.apache.org>
>
>
> Hi, Lahir,
>
> I succeeded to use ForEach component with attaching 'Input' component in
> front of ForEach componet with comma separated value(i.e "a,b,c") as input
> like in attached figure. ForEach component works well with Input component
> with comma separate value.
>
> Now I want to mimic what Input component does within my script as in another
> figure. This is required to run ForEach on generated data not on data
> inputed from user.
>
> I tired several ways to generate a input for ForEach component in my script
> including what you suggested in previous mail, but I couldn't. Please help
> me to solve this problem.
>
> Thank you,
> Heejoon.
>
>
> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>
> wrote:
>>
>> Following script will be useful for you..
>>
>> echo $@
>> echo $*
>> echo $#
>> for arg; do
>> printf '%s\n' "$arg"
>> done
>>
>>
>> Lahir
>>
>>
>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>>>
>>> Hi Lahiru,
>>>
>>> Thank you for your answer.
>>>
>>> By the way, how to create string array in shell script, so that I can
>>> pass it to ForEach component?
>>>
>>> Thank you,
>>> Heejoon.
>>>
>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>>>
>>> Hi Heejoon,
>>>
>>> We take string array as input but we invoke the script with each element
>>> of the string array multiple times, in Foreach we don't pass whole string
>>> array to the application script at once. So you really don't have to handle
>>> a string array in your script.
>>>
>>> Lahiru
>>>
>>>
>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>
>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I have quick question about Airavata For-Each component.
>>>>
>>>> It seems that ForEach component takes String array as an input and pass
>>>> them to next component one by one.
>>>>
>>>> My question is how to create string array in shell script to pass it to
>>>> ForEach component?
>>>>
>>>> I normally generate string result like
>>>>
>>>> echo "output=xxx"
>>>>
>>>> at the end of script and connect it as an input of next component in
>>>> Airavata workflow to pass the result and let next component know the input
>>>> is arrived.
>>>>
>>>> Thanks,
>>>> Heejoon.
>>>
>>>
>>>
>>>
>>> --
>>> System Analyst Programmer
>>> PTI Lab
>>> Indiana University
>>
>>
>>
>>
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University
>
>
>
> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at 3.11.31
> PM.png>
>
>
--
Chathura Herath Ph.D
http://people.apache.org/~chathura/
http://chathurah.blogspot.com/
Re: Question for Airavata For-Each component
Posted by Heejoon Chae <ic...@gmail.com>.
Hi, Chathura,
This might be correct one. I also attached slightly modified first script.
<ns:receivedResult infoModelVersion="2.6"
xmlns:ns="http://airavata.apache.org/schemas/wft/2011/08">
<ns:notificationSource
ns:serviceID="Workflow1_550b44fa_35b7_4c27_b194_eca8c15365ae"
ns:experimentID="Workflow1_550b44fa-35b7-4c27-b194-eca8c15365ae" />
<ns:timestamp>2013-07-12T10:56:00.525+09:00</ns:timestamp>
<ns:description>output=null</ns:description>
<ns:annotation />
<ns:result>
<ns:body>
<ns1:invokeResponse xmlns:ns1="http://ws.apache.org/axis2/xsd">
<ns:output xsi:type="ns:StringArrayType"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ns="http://airavata.apache.org/schemas/gfac/2012/12" />
</ns1:invokeResponse>
</ns:body>
</ns:result>
<ns:responder ns:serviceID="string_array_test_invoke"
ns:workflowID="Workflow1_550b44fa_35b7_4c27_b194_eca8c15365ae"
ns:workflowTimestep="0" ns:workflowNodeID="string_array_test_invoke" />
</ns:receivedResult>
Thank you,
Heejoon.
On Fri, Jul 12, 2013 at 10:37 AM, Chathura Herath <ch...@cs.indiana.edu>wrote:
> Forwarding to list
>
>
> ---------- Forwarded message ----------
> From: Chathura Herath <ch...@cs.indiana.edu>
> Date: Mon, Jul 1, 2013 at 9:11 PM
> Subject: Re: Question for Airavata For-Each component
> To: Suresh Marru <sm...@apache.org>
>
>
> Heejoon,
>
> This isnt the msg we are looking for. This looks like the first msg.
> There should be another msg something like response received which
> should have an array of output files.
>
> On Mon, Jul 1, 2013 at 9:49 AM, Suresh Marru <sm...@apache.org> wrote:
> > Saminda,
> >
> > Chathura preciously knows what needs to be done, can you work with him
> and help Heejoon.
> >
> > Suresh
> >
> > On Jul 1, 2013, at 12:27 AM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> >
> >> Yes. It just passed the location of data file that I put to second
> component.
> >> Here is the xml from first input component.
> >>
> >> <ns:workflowInvoked infoModelVersion="2.6"
> >> xmlns:ns="http://airavata.apache.org/schemas/wft/2011/08">
> >> <ns:notificationSource
> ns:serviceID="Workflow1_1e933fac_1c73_4fdf_90a1_3bb3b310d188"
> ns:experimentID="Workflow1_1e933fac-1c73-4fdf-90a1-3bb3b310d188" />
> >> <ns:timestamp>2013-07-01T10:22:30.198+09:00</ns:timestamp>
> >>
> <ns:description>input_file_list=/tmp/input_file_list.txt</ns:description>
> >> <ns:annotation />
> >> <ns:initiator
> ns:serviceID="Workflow1_1e933fac_1c73_4fdf_90a1_3bb3b310d188" />
> >> </ns:workflowInvoked>
> >>
> >> Thank you,
> >> Heejoon.
> >>
> >>
> >> On Mon, Jul 1, 2013 at 1:14 PM, Chathura Herath <ch...@cs.indiana.edu>
> wrote:
> >> I believe Xbaya runs your first component without error, is it not? I
> >> was asking for the output xml of the first service . You may find this
> >> by opening the notifications you received. It should have the array
> >> elements.
> >>
> >> On Sun, Jun 30, 2013 at 9:34 PM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> >> > Hi, Chathura,
> >> >
> >> > For some reason, previous xwf gives errors, so I am re-attaching the
> scripts
> >> > and xwf.
> >> > I also attached wsdl from the first component
> >> >
> >> > Thank you!,
> >> > Heejoon.
> >> >
> >> >
> >> >
> >> > On Sun, Jun 30, 2013 at 2:01 PM, Chathura Herath <
> cherath@cs.indiana.edu>
> >> > wrote:
> >> >>
> >> >> send me the result xml from your first component as well
> >> >>
> >> >> On Sat, Jun 29, 2013 at 11:26 PM, Heejoon Chae <
> heechae@cs.indiana.edu>
> >> >> wrote:
> >> >> > Hi, Chathura,
> >> >> >
> >> >> > I am attaching simplified xwf and related scripts.
> >> >> >
> >> >> > Basically if user input filename which contains list of file names,
> >> >> > 'string_array_test.sh' script generates input for 'foreach
> component'.
> >> >> > Then, 'local_echo.sh' will do something on each passed file name.
> >> >> >
> >> >> > Thank you for taking your time.
> >> >> > Heejoon.
> >> >> >
> >> >> >
> >> >> > On Sun, Jun 30, 2013 at 12:24 AM, Chathura Herath
> >> >> > <ch...@cs.indiana.edu>
> >> >> > wrote:
> >> >> >>
> >> >> >> Two questions.
> >> >> >> 1) your services are they public and can i access them.
> >> >> >> 2) Can you send us your xwf file.
> >> >> >>
> >> >> >> If yes to both of the above we can actually load your workflow and
> >> >> >> debug
> >> >> >> this.
> >> >> >>
> >> >> >> If not;
> >> >> >> I looked at the code, the exception you get is because xbaya
> couldnt
> >> >> >> find the element array. From what i see in the code you need one
> more
> >> >> >> wrapper element. This is the wrapper element you get when u
> define in
> >> >> >> the wsdl an element for max occurs unbounded.
> >> >> >>
> >> >> >> I don't want to disappoint you with many trial and errors. you
> can try
> >> >> >> this if not give us the services and the xwf so one of us can
> take a
> >> >> >> look.
> >> >> >>
> >> >> >>
> >> >> >> <ComponentOneOrWhateverOutputParamname>
> >> >> >> <somewrapperelement>
> >> >> >> <val>a</val>
> >> >> >> <val>b</val>
> >> >> >> </somewrapperelement>
> >> >> >> ..
> >> >> >> </ComponentOneOrWhateverOutputParamname>
> >> >> >>
> >> >> >> On Sat, Jun 29, 2013 at 3:38 AM, Heejoon Chae <
> heechae@cs.indiana.edu>
> >> >> >> wrote:
> >> >> >> > Hi, Suresh,
> >> >> >> >
> >> >> >> > Any updates on this?
> >> >> >> >
> >> >> >> > By the way, can you also let me know how I cloud run for-each
> >> >> >> > component
> >> >> >> > on
> >> >> >> > files? Actually, what I want to do is to run my script on
> several
> >> >> >> > input
> >> >> >> > files(let's say 100 inputs), but actual input files'
> informations are
> >> >> >> > listed
> >> >> >> > on a single file instead of getting all the input one by one.
> >> >> >> >
> >> >> >> > So, the input is single file containing all the input file
> >> >> >> > informations
> >> >> >> > looks like,
> >> >> >> >
> >> >> >> > ...
> >> >> >> > http://xxx,xxx,xxx/input1.txt
> >> >> >> > http://xxx,xxx,xxx/input2.txt
> >> >> >> > http://xxx,xxx,xxx/input3.txt
> >> >> >> > ...
> >> >> >> >
> >> >> >> > Thanks!
> >> >> >> > Heejoon.
> >> >> >> >
> >> >> >> >
> >> >> >> > On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <
> smarru@apache.org>
> >> >> >> > wrote:
> >> >> >> >>
> >> >> >> >> Hi Heejoon,
> >> >> >> >>
> >> >> >> >> Normally we use for-each on files, so the way to handle that
> is let
> >> >> >> >> all
> >> >> >> >> the output files go into a outputData directory and iterate
> over
> >> >> >> >> them.
> >> >> >> >> But I
> >> >> >> >> see what you are doing here. Let me try this and get back to
> you
> >> >> >> >> within
> >> >> >> >> the
> >> >> >> >> next day.
> >> >> >> >>
> >> >> >> >> Thanks for your patience,
> >> >> >> >> Suresh
> >> >> >> >>
> >> >> >> >> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <
> heechae@cs.indiana.edu>
> >> >> >> >> wrote:
> >> >> >> >>
> >> >> >> >> Hi, Airavata users,
> >> >> >> >>
> >> >> >> >> Do you have any advises or experiences for my previous mail
> about
> >> >> >> >> ForEach
> >> >> >> >> component?
> >> >> >> >> I am still struggling with generating input for ForEach
> component in
> >> >> >> >> my
> >> >> >> >> script :/
> >> >> >> >> Any comment will be helpful to me to use the component :)
> >> >> >> >>
> >> >> >> >> Thank you!
> >> >> >> >>
> >> >> >> >> ---------- Forwarded message ----------
> >> >> >> >> From: Heejoon Chae <he...@cs.indiana.edu>
> >> >> >> >> Date: Fri, Jun 7, 2013 at 5:44 PM
> >> >> >> >> Subject: Re: Question for Airavata For-Each component
> >> >> >> >> To: "users@airavata.apache.org" <us...@airavata.apache.org>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> Hi, Lahir,
> >> >> >> >>
> >> >> >> >> I succeeded to use ForEach component with attaching 'Input'
> >> >> >> >> component
> >> >> >> >> in
> >> >> >> >> front of ForEach componet with comma separated value(i.e
> "a,b,c") as
> >> >> >> >> input
> >> >> >> >> like in attached figure. ForEach component works well with
> Input
> >> >> >> >> component
> >> >> >> >> with comma separate value.
> >> >> >> >>
> >> >> >> >> Now I want to mimic what Input component does within my script
> as in
> >> >> >> >> another figure. This is required to run ForEach on generated
> data
> >> >> >> >> not
> >> >> >> >> on
> >> >> >> >> data inputed from user.
> >> >> >> >>
> >> >> >> >> I tired several ways to generate a input for ForEach component
> in my
> >> >> >> >> script including what you suggested in previous mail, but I
> >> >> >> >> couldn't.
> >> >> >> >> Please
> >> >> >> >> help me to solve this problem.
> >> >> >> >>
> >> >> >> >> Thank you,
> >> >> >> >> Heejoon.
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake
> >> >> >> >> <gl...@gmail.com>
> >> >> >> >> wrote:
> >> >> >> >>>
> >> >> >> >>> Following script will be useful for you..
> >> >> >> >>>
> >> >> >> >>> echo $@
> >> >> >> >>> echo $*
> >> >> >> >>> echo $#
> >> >> >> >>> for arg; do
> >> >> >> >>> printf '%s\n' "$arg"
> >> >> >> >>> done
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> Lahir
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
> >> >> >> >>>>
> >> >> >> >>>> Hi Lahiru,
> >> >> >> >>>>
> >> >> >> >>>> Thank you for your answer.
> >> >> >> >>>>
> >> >> >> >>>> By the way, how to create string array in shell script, so
> that I
> >> >> >> >>>> can
> >> >> >> >>>> pass it to ForEach component?
> >> >> >> >>>>
> >> >> >> >>>> Thank you,
> >> >> >> >>>> Heejoon.
> >> >> >> >>>>
> >> >> >> >>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com>
> 작성:
> >> >> >> >>>>
> >> >> >> >>>> Hi Heejoon,
> >> >> >> >>>>
> >> >> >> >>>> We take string array as input but we invoke the script with
> each
> >> >> >> >>>> element
> >> >> >> >>>> of the string array multiple times, in Foreach we don't pass
> whole
> >> >> >> >>>> string
> >> >> >> >>>> array to the application script at once. So you really don't
> have
> >> >> >> >>>> to
> >> >> >> >>>> handle
> >> >> >> >>>> a string array in your script.
> >> >> >> >>>>
> >> >> >> >>>> Lahiru
> >> >> >> >>>>
> >> >> >> >>>>
> >> >> >> >>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae
> >> >> >> >>>> <he...@cs.indiana.edu>
> >> >> >> >>>> wrote:
> >> >> >> >>>>>
> >> >> >> >>>>> Hi,
> >> >> >> >>>>>
> >> >> >> >>>>> I have quick question about Airavata For-Each component.
> >> >> >> >>>>>
> >> >> >> >>>>> It seems that ForEach component takes String array as an
> input
> >> >> >> >>>>> and
> >> >> >> >>>>> pass
> >> >> >> >>>>> them to next component one by one.
> >> >> >> >>>>>
> >> >> >> >>>>> My question is how to create string array in shell script
> to pass
> >> >> >> >>>>> it
> >> >> >> >>>>> to
> >> >> >> >>>>> ForEach component?
> >> >> >> >>>>>
> >> >> >> >>>>> I normally generate string result like
> >> >> >> >>>>>
> >> >> >> >>>>> echo "output=xxx"
> >> >> >> >>>>>
> >> >> >> >>>>> at the end of script and connect it as an input of next
> component
> >> >> >> >>>>> in
> >> >> >> >>>>> Airavata workflow to pass the result and let next component
> know
> >> >> >> >>>>> the
> >> >> >> >>>>> input
> >> >> >> >>>>> is arrived.
> >> >> >> >>>>>
> >> >> >> >>>>> Thanks,
> >> >> >> >>>>> Heejoon.
> >> >> >> >>>>
> >> >> >> >>>>
> >> >> >> >>>>
> >> >> >> >>>>
> >> >> >> >>>> --
> >> >> >> >>>> System Analyst Programmer
> >> >> >> >>>> PTI Lab
> >> >> >> >>>> Indiana University
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> --
> >> >> >> >>> System Analyst Programmer
> >> >> >> >>> PTI Lab
> >> >> >> >>> Indiana University
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot
> 2013-06-07 at
> >> >> >> >> 3.11.31 PM.png>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Chathura Herath Ph.D
> >> >> >> http://people.apache.org/~chathura/
> >> >> >> http://chathurah.blogspot.com/
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Chathura Herath Ph.D
> >> >> http://people.apache.org/~chathura/
> >> >> http://chathurah.blogspot.com/
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Chathura Herath Ph.D
> >> http://people.apache.org/~chathura/
> >> http://chathurah.blogspot.com/
> >>
> >
>
>
>
> --
> Chathura Herath Ph.D
> http://people.apache.org/~chathura/
> http://chathurah.blogspot.com/
>
>
> --
> Chathura Herath Ph.D
> http://people.apache.org/~chathura/
> http://chathurah.blogspot.com/
>
Fwd: Question for Airavata For-Each component
Posted by Chathura Herath <ch...@cs.indiana.edu>.
Forwarding to list
---------- Forwarded message ----------
From: Chathura Herath <ch...@cs.indiana.edu>
Date: Mon, Jul 1, 2013 at 9:11 PM
Subject: Re: Question for Airavata For-Each component
To: Suresh Marru <sm...@apache.org>
Heejoon,
This isnt the msg we are looking for. This looks like the first msg.
There should be another msg something like response received which
should have an array of output files.
On Mon, Jul 1, 2013 at 9:49 AM, Suresh Marru <sm...@apache.org> wrote:
> Saminda,
>
> Chathura preciously knows what needs to be done, can you work with him and help Heejoon.
>
> Suresh
>
> On Jul 1, 2013, at 12:27 AM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>
>> Yes. It just passed the location of data file that I put to second component.
>> Here is the xml from first input component.
>>
>> <ns:workflowInvoked infoModelVersion="2.6"
>> xmlns:ns="http://airavata.apache.org/schemas/wft/2011/08">
>> <ns:notificationSource ns:serviceID="Workflow1_1e933fac_1c73_4fdf_90a1_3bb3b310d188" ns:experimentID="Workflow1_1e933fac-1c73-4fdf-90a1-3bb3b310d188" />
>> <ns:timestamp>2013-07-01T10:22:30.198+09:00</ns:timestamp>
>> <ns:description>input_file_list=/tmp/input_file_list.txt</ns:description>
>> <ns:annotation />
>> <ns:initiator ns:serviceID="Workflow1_1e933fac_1c73_4fdf_90a1_3bb3b310d188" />
>> </ns:workflowInvoked>
>>
>> Thank you,
>> Heejoon.
>>
>>
>> On Mon, Jul 1, 2013 at 1:14 PM, Chathura Herath <ch...@cs.indiana.edu> wrote:
>> I believe Xbaya runs your first component without error, is it not? I
>> was asking for the output xml of the first service . You may find this
>> by opening the notifications you received. It should have the array
>> elements.
>>
>> On Sun, Jun 30, 2013 at 9:34 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>> > Hi, Chathura,
>> >
>> > For some reason, previous xwf gives errors, so I am re-attaching the scripts
>> > and xwf.
>> > I also attached wsdl from the first component
>> >
>> > Thank you!,
>> > Heejoon.
>> >
>> >
>> >
>> > On Sun, Jun 30, 2013 at 2:01 PM, Chathura Herath <ch...@cs.indiana.edu>
>> > wrote:
>> >>
>> >> send me the result xml from your first component as well
>> >>
>> >> On Sat, Jun 29, 2013 at 11:26 PM, Heejoon Chae <he...@cs.indiana.edu>
>> >> wrote:
>> >> > Hi, Chathura,
>> >> >
>> >> > I am attaching simplified xwf and related scripts.
>> >> >
>> >> > Basically if user input filename which contains list of file names,
>> >> > 'string_array_test.sh' script generates input for 'foreach component'.
>> >> > Then, 'local_echo.sh' will do something on each passed file name.
>> >> >
>> >> > Thank you for taking your time.
>> >> > Heejoon.
>> >> >
>> >> >
>> >> > On Sun, Jun 30, 2013 at 12:24 AM, Chathura Herath
>> >> > <ch...@cs.indiana.edu>
>> >> > wrote:
>> >> >>
>> >> >> Two questions.
>> >> >> 1) your services are they public and can i access them.
>> >> >> 2) Can you send us your xwf file.
>> >> >>
>> >> >> If yes to both of the above we can actually load your workflow and
>> >> >> debug
>> >> >> this.
>> >> >>
>> >> >> If not;
>> >> >> I looked at the code, the exception you get is because xbaya couldnt
>> >> >> find the element array. From what i see in the code you need one more
>> >> >> wrapper element. This is the wrapper element you get when u define in
>> >> >> the wsdl an element for max occurs unbounded.
>> >> >>
>> >> >> I don't want to disappoint you with many trial and errors. you can try
>> >> >> this if not give us the services and the xwf so one of us can take a
>> >> >> look.
>> >> >>
>> >> >>
>> >> >> <ComponentOneOrWhateverOutputParamname>
>> >> >> <somewrapperelement>
>> >> >> <val>a</val>
>> >> >> <val>b</val>
>> >> >> </somewrapperelement>
>> >> >> ..
>> >> >> </ComponentOneOrWhateverOutputParamname>
>> >> >>
>> >> >> On Sat, Jun 29, 2013 at 3:38 AM, Heejoon Chae <he...@cs.indiana.edu>
>> >> >> wrote:
>> >> >> > Hi, Suresh,
>> >> >> >
>> >> >> > Any updates on this?
>> >> >> >
>> >> >> > By the way, can you also let me know how I cloud run for-each
>> >> >> > component
>> >> >> > on
>> >> >> > files? Actually, what I want to do is to run my script on several
>> >> >> > input
>> >> >> > files(let's say 100 inputs), but actual input files' informations are
>> >> >> > listed
>> >> >> > on a single file instead of getting all the input one by one.
>> >> >> >
>> >> >> > So, the input is single file containing all the input file
>> >> >> > informations
>> >> >> > looks like,
>> >> >> >
>> >> >> > ...
>> >> >> > http://xxx,xxx,xxx/input1.txt
>> >> >> > http://xxx,xxx,xxx/input2.txt
>> >> >> > http://xxx,xxx,xxx/input3.txt
>> >> >> > ...
>> >> >> >
>> >> >> > Thanks!
>> >> >> > Heejoon.
>> >> >> >
>> >> >> >
>> >> >> > On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Hi Heejoon,
>> >> >> >>
>> >> >> >> Normally we use for-each on files, so the way to handle that is let
>> >> >> >> all
>> >> >> >> the output files go into a outputData directory and iterate over
>> >> >> >> them.
>> >> >> >> But I
>> >> >> >> see what you are doing here. Let me try this and get back to you
>> >> >> >> within
>> >> >> >> the
>> >> >> >> next day.
>> >> >> >>
>> >> >> >> Thanks for your patience,
>> >> >> >> Suresh
>> >> >> >>
>> >> >> >> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu>
>> >> >> >> wrote:
>> >> >> >>
>> >> >> >> Hi, Airavata users,
>> >> >> >>
>> >> >> >> Do you have any advises or experiences for my previous mail about
>> >> >> >> ForEach
>> >> >> >> component?
>> >> >> >> I am still struggling with generating input for ForEach component in
>> >> >> >> my
>> >> >> >> script :/
>> >> >> >> Any comment will be helpful to me to use the component :)
>> >> >> >>
>> >> >> >> Thank you!
>> >> >> >>
>> >> >> >> ---------- Forwarded message ----------
>> >> >> >> From: Heejoon Chae <he...@cs.indiana.edu>
>> >> >> >> Date: Fri, Jun 7, 2013 at 5:44 PM
>> >> >> >> Subject: Re: Question for Airavata For-Each component
>> >> >> >> To: "users@airavata.apache.org" <us...@airavata.apache.org>
>> >> >> >>
>> >> >> >>
>> >> >> >> Hi, Lahir,
>> >> >> >>
>> >> >> >> I succeeded to use ForEach component with attaching 'Input'
>> >> >> >> component
>> >> >> >> in
>> >> >> >> front of ForEach componet with comma separated value(i.e "a,b,c") as
>> >> >> >> input
>> >> >> >> like in attached figure. ForEach component works well with Input
>> >> >> >> component
>> >> >> >> with comma separate value.
>> >> >> >>
>> >> >> >> Now I want to mimic what Input component does within my script as in
>> >> >> >> another figure. This is required to run ForEach on generated data
>> >> >> >> not
>> >> >> >> on
>> >> >> >> data inputed from user.
>> >> >> >>
>> >> >> >> I tired several ways to generate a input for ForEach component in my
>> >> >> >> script including what you suggested in previous mail, but I
>> >> >> >> couldn't.
>> >> >> >> Please
>> >> >> >> help me to solve this problem.
>> >> >> >>
>> >> >> >> Thank you,
>> >> >> >> Heejoon.
>> >> >> >>
>> >> >> >>
>> >> >> >> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake
>> >> >> >> <gl...@gmail.com>
>> >> >> >> wrote:
>> >> >> >>>
>> >> >> >>> Following script will be useful for you..
>> >> >> >>>
>> >> >> >>> echo $@
>> >> >> >>> echo $*
>> >> >> >>> echo $#
>> >> >> >>> for arg; do
>> >> >> >>> printf '%s\n' "$arg"
>> >> >> >>> done
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> Lahir
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>> >> >> >>>>
>> >> >> >>>> Hi Lahiru,
>> >> >> >>>>
>> >> >> >>>> Thank you for your answer.
>> >> >> >>>>
>> >> >> >>>> By the way, how to create string array in shell script, so that I
>> >> >> >>>> can
>> >> >> >>>> pass it to ForEach component?
>> >> >> >>>>
>> >> >> >>>> Thank you,
>> >> >> >>>> Heejoon.
>> >> >> >>>>
>> >> >> >>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>> >> >> >>>>
>> >> >> >>>> Hi Heejoon,
>> >> >> >>>>
>> >> >> >>>> We take string array as input but we invoke the script with each
>> >> >> >>>> element
>> >> >> >>>> of the string array multiple times, in Foreach we don't pass whole
>> >> >> >>>> string
>> >> >> >>>> array to the application script at once. So you really don't have
>> >> >> >>>> to
>> >> >> >>>> handle
>> >> >> >>>> a string array in your script.
>> >> >> >>>>
>> >> >> >>>> Lahiru
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae
>> >> >> >>>> <he...@cs.indiana.edu>
>> >> >> >>>> wrote:
>> >> >> >>>>>
>> >> >> >>>>> Hi,
>> >> >> >>>>>
>> >> >> >>>>> I have quick question about Airavata For-Each component.
>> >> >> >>>>>
>> >> >> >>>>> It seems that ForEach component takes String array as an input
>> >> >> >>>>> and
>> >> >> >>>>> pass
>> >> >> >>>>> them to next component one by one.
>> >> >> >>>>>
>> >> >> >>>>> My question is how to create string array in shell script to pass
>> >> >> >>>>> it
>> >> >> >>>>> to
>> >> >> >>>>> ForEach component?
>> >> >> >>>>>
>> >> >> >>>>> I normally generate string result like
>> >> >> >>>>>
>> >> >> >>>>> echo "output=xxx"
>> >> >> >>>>>
>> >> >> >>>>> at the end of script and connect it as an input of next component
>> >> >> >>>>> in
>> >> >> >>>>> Airavata workflow to pass the result and let next component know
>> >> >> >>>>> the
>> >> >> >>>>> input
>> >> >> >>>>> is arrived.
>> >> >> >>>>>
>> >> >> >>>>> Thanks,
>> >> >> >>>>> Heejoon.
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> --
>> >> >> >>>> System Analyst Programmer
>> >> >> >>>> PTI Lab
>> >> >> >>>> Indiana University
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> --
>> >> >> >>> System Analyst Programmer
>> >> >> >>> PTI Lab
>> >> >> >>> Indiana University
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
>> >> >> >> 3.11.31 PM.png>
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Chathura Herath Ph.D
>> >> >> http://people.apache.org/~chathura/
>> >> >> http://chathurah.blogspot.com/
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Chathura Herath Ph.D
>> >> http://people.apache.org/~chathura/
>> >> http://chathurah.blogspot.com/
>> >
>> >
>>
>>
>>
>> --
>> Chathura Herath Ph.D
>> http://people.apache.org/~chathura/
>> http://chathurah.blogspot.com/
>>
>
--
Chathura Herath Ph.D
http://people.apache.org/~chathura/
http://chathurah.blogspot.com/
--
Chathura Herath Ph.D
http://people.apache.org/~chathura/
http://chathurah.blogspot.com/
Re: Question for Airavata For-Each component
Posted by Heejoon Chae <he...@cs.indiana.edu>.
Yes. It just passed the location of data file that I put to second
component.
Here is the xml from first input component.
<ns:workflowInvoked infoModelVersion="2.6"
xmlns:ns="http://airavata.apache.org/schemas/wft/2011/08">
<ns:notificationSource
ns:serviceID="Workflow1_1e933fac_1c73_4fdf_90a1_3bb3b310d188"
ns:experimentID="Workflow1_1e933fac-1c73-4fdf-90a1-3bb3b310d188" />
<ns:timestamp>2013-07-01T10:22:30.198+09:00</ns:timestamp>
<ns:description>input_file_list=/tmp/input_file_list.txt</ns:description>
<ns:annotation />
<ns:initiator
ns:serviceID="Workflow1_1e933fac_1c73_4fdf_90a1_3bb3b310d188" />
</ns:workflowInvoked>
Thank you,
Heejoon.
On Mon, Jul 1, 2013 at 1:14 PM, Chathura Herath <ch...@cs.indiana.edu>wrote:
> I believe Xbaya runs your first component without error, is it not? I
> was asking for the output xml of the first service . You may find this
> by opening the notifications you received. It should have the array
> elements.
>
> On Sun, Jun 30, 2013 at 9:34 PM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> > Hi, Chathura,
> >
> > For some reason, previous xwf gives errors, so I am re-attaching the
> scripts
> > and xwf.
> > I also attached wsdl from the first component
> >
> > Thank you!,
> > Heejoon.
> >
> >
> >
> > On Sun, Jun 30, 2013 at 2:01 PM, Chathura Herath <cherath@cs.indiana.edu
> >
> > wrote:
> >>
> >> send me the result xml from your first component as well
> >>
> >> On Sat, Jun 29, 2013 at 11:26 PM, Heejoon Chae <he...@cs.indiana.edu>
> >> wrote:
> >> > Hi, Chathura,
> >> >
> >> > I am attaching simplified xwf and related scripts.
> >> >
> >> > Basically if user input filename which contains list of file names,
> >> > 'string_array_test.sh' script generates input for 'foreach component'.
> >> > Then, 'local_echo.sh' will do something on each passed file name.
> >> >
> >> > Thank you for taking your time.
> >> > Heejoon.
> >> >
> >> >
> >> > On Sun, Jun 30, 2013 at 12:24 AM, Chathura Herath
> >> > <ch...@cs.indiana.edu>
> >> > wrote:
> >> >>
> >> >> Two questions.
> >> >> 1) your services are they public and can i access them.
> >> >> 2) Can you send us your xwf file.
> >> >>
> >> >> If yes to both of the above we can actually load your workflow and
> >> >> debug
> >> >> this.
> >> >>
> >> >> If not;
> >> >> I looked at the code, the exception you get is because xbaya couldnt
> >> >> find the element array. From what i see in the code you need one more
> >> >> wrapper element. This is the wrapper element you get when u define in
> >> >> the wsdl an element for max occurs unbounded.
> >> >>
> >> >> I don't want to disappoint you with many trial and errors. you can
> try
> >> >> this if not give us the services and the xwf so one of us can take a
> >> >> look.
> >> >>
> >> >>
> >> >> <ComponentOneOrWhateverOutputParamname>
> >> >> <somewrapperelement>
> >> >> <val>a</val>
> >> >> <val>b</val>
> >> >> </somewrapperelement>
> >> >> ..
> >> >> </ComponentOneOrWhateverOutputParamname>
> >> >>
> >> >> On Sat, Jun 29, 2013 at 3:38 AM, Heejoon Chae <
> heechae@cs.indiana.edu>
> >> >> wrote:
> >> >> > Hi, Suresh,
> >> >> >
> >> >> > Any updates on this?
> >> >> >
> >> >> > By the way, can you also let me know how I cloud run for-each
> >> >> > component
> >> >> > on
> >> >> > files? Actually, what I want to do is to run my script on several
> >> >> > input
> >> >> > files(let's say 100 inputs), but actual input files' informations
> are
> >> >> > listed
> >> >> > on a single file instead of getting all the input one by one.
> >> >> >
> >> >> > So, the input is single file containing all the input file
> >> >> > informations
> >> >> > looks like,
> >> >> >
> >> >> > ...
> >> >> > http://xxx,xxx,xxx/input1.txt
> >> >> > http://xxx,xxx,xxx/input2.txt
> >> >> > http://xxx,xxx,xxx/input3.txt
> >> >> > ...
> >> >> >
> >> >> > Thanks!
> >> >> > Heejoon.
> >> >> >
> >> >> >
> >> >> > On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org>
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi Heejoon,
> >> >> >>
> >> >> >> Normally we use for-each on files, so the way to handle that is
> let
> >> >> >> all
> >> >> >> the output files go into a outputData directory and iterate over
> >> >> >> them.
> >> >> >> But I
> >> >> >> see what you are doing here. Let me try this and get back to you
> >> >> >> within
> >> >> >> the
> >> >> >> next day.
> >> >> >>
> >> >> >> Thanks for your patience,
> >> >> >> Suresh
> >> >> >>
> >> >> >> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <
> heechae@cs.indiana.edu>
> >> >> >> wrote:
> >> >> >>
> >> >> >> Hi, Airavata users,
> >> >> >>
> >> >> >> Do you have any advises or experiences for my previous mail about
> >> >> >> ForEach
> >> >> >> component?
> >> >> >> I am still struggling with generating input for ForEach component
> in
> >> >> >> my
> >> >> >> script :/
> >> >> >> Any comment will be helpful to me to use the component :)
> >> >> >>
> >> >> >> Thank you!
> >> >> >>
> >> >> >> ---------- Forwarded message ----------
> >> >> >> From: Heejoon Chae <he...@cs.indiana.edu>
> >> >> >> Date: Fri, Jun 7, 2013 at 5:44 PM
> >> >> >> Subject: Re: Question for Airavata For-Each component
> >> >> >> To: "users@airavata.apache.org" <us...@airavata.apache.org>
> >> >> >>
> >> >> >>
> >> >> >> Hi, Lahir,
> >> >> >>
> >> >> >> I succeeded to use ForEach component with attaching 'Input'
> >> >> >> component
> >> >> >> in
> >> >> >> front of ForEach componet with comma separated value(i.e "a,b,c")
> as
> >> >> >> input
> >> >> >> like in attached figure. ForEach component works well with Input
> >> >> >> component
> >> >> >> with comma separate value.
> >> >> >>
> >> >> >> Now I want to mimic what Input component does within my script as
> in
> >> >> >> another figure. This is required to run ForEach on generated data
> >> >> >> not
> >> >> >> on
> >> >> >> data inputed from user.
> >> >> >>
> >> >> >> I tired several ways to generate a input for ForEach component in
> my
> >> >> >> script including what you suggested in previous mail, but I
> >> >> >> couldn't.
> >> >> >> Please
> >> >> >> help me to solve this problem.
> >> >> >>
> >> >> >> Thank you,
> >> >> >> Heejoon.
> >> >> >>
> >> >> >>
> >> >> >> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake
> >> >> >> <gl...@gmail.com>
> >> >> >> wrote:
> >> >> >>>
> >> >> >>> Following script will be useful for you..
> >> >> >>>
> >> >> >>> echo $@
> >> >> >>> echo $*
> >> >> >>> echo $#
> >> >> >>> for arg; do
> >> >> >>> printf '%s\n' "$arg"
> >> >> >>> done
> >> >> >>>
> >> >> >>>
> >> >> >>> Lahir
> >> >> >>>
> >> >> >>>
> >> >> >>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
> >> >> >>>>
> >> >> >>>> Hi Lahiru,
> >> >> >>>>
> >> >> >>>> Thank you for your answer.
> >> >> >>>>
> >> >> >>>> By the way, how to create string array in shell script, so that
> I
> >> >> >>>> can
> >> >> >>>> pass it to ForEach component?
> >> >> >>>>
> >> >> >>>> Thank you,
> >> >> >>>> Heejoon.
> >> >> >>>>
> >> >> >>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
> >> >> >>>>
> >> >> >>>> Hi Heejoon,
> >> >> >>>>
> >> >> >>>> We take string array as input but we invoke the script with each
> >> >> >>>> element
> >> >> >>>> of the string array multiple times, in Foreach we don't pass
> whole
> >> >> >>>> string
> >> >> >>>> array to the application script at once. So you really don't
> have
> >> >> >>>> to
> >> >> >>>> handle
> >> >> >>>> a string array in your script.
> >> >> >>>>
> >> >> >>>> Lahiru
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae
> >> >> >>>> <he...@cs.indiana.edu>
> >> >> >>>> wrote:
> >> >> >>>>>
> >> >> >>>>> Hi,
> >> >> >>>>>
> >> >> >>>>> I have quick question about Airavata For-Each component.
> >> >> >>>>>
> >> >> >>>>> It seems that ForEach component takes String array as an input
> >> >> >>>>> and
> >> >> >>>>> pass
> >> >> >>>>> them to next component one by one.
> >> >> >>>>>
> >> >> >>>>> My question is how to create string array in shell script to
> pass
> >> >> >>>>> it
> >> >> >>>>> to
> >> >> >>>>> ForEach component?
> >> >> >>>>>
> >> >> >>>>> I normally generate string result like
> >> >> >>>>>
> >> >> >>>>> echo "output=xxx"
> >> >> >>>>>
> >> >> >>>>> at the end of script and connect it as an input of next
> component
> >> >> >>>>> in
> >> >> >>>>> Airavata workflow to pass the result and let next component
> know
> >> >> >>>>> the
> >> >> >>>>> input
> >> >> >>>>> is arrived.
> >> >> >>>>>
> >> >> >>>>> Thanks,
> >> >> >>>>> Heejoon.
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> --
> >> >> >>>> System Analyst Programmer
> >> >> >>>> PTI Lab
> >> >> >>>> Indiana University
> >> >> >>>
> >> >> >>>
> >> >> >>>
> >> >> >>>
> >> >> >>> --
> >> >> >>> System Analyst Programmer
> >> >> >>> PTI Lab
> >> >> >>> Indiana University
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07
> at
> >> >> >> 3.11.31 PM.png>
> >> >> >>
> >> >> >>
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Chathura Herath Ph.D
> >> >> http://people.apache.org/~chathura/
> >> >> http://chathurah.blogspot.com/
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Chathura Herath Ph.D
> >> http://people.apache.org/~chathura/
> >> http://chathurah.blogspot.com/
> >
> >
>
>
>
> --
> Chathura Herath Ph.D
> http://people.apache.org/~chathura/
> http://chathurah.blogspot.com/
>
Re: Question for Airavata For-Each component
Posted by Chathura Herath <ch...@cs.indiana.edu>.
I believe Xbaya runs your first component without error, is it not? I
was asking for the output xml of the first service . You may find this
by opening the notifications you received. It should have the array
elements.
On Sun, Jun 30, 2013 at 9:34 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
> Hi, Chathura,
>
> For some reason, previous xwf gives errors, so I am re-attaching the scripts
> and xwf.
> I also attached wsdl from the first component
>
> Thank you!,
> Heejoon.
>
>
>
> On Sun, Jun 30, 2013 at 2:01 PM, Chathura Herath <ch...@cs.indiana.edu>
> wrote:
>>
>> send me the result xml from your first component as well
>>
>> On Sat, Jun 29, 2013 at 11:26 PM, Heejoon Chae <he...@cs.indiana.edu>
>> wrote:
>> > Hi, Chathura,
>> >
>> > I am attaching simplified xwf and related scripts.
>> >
>> > Basically if user input filename which contains list of file names,
>> > 'string_array_test.sh' script generates input for 'foreach component'.
>> > Then, 'local_echo.sh' will do something on each passed file name.
>> >
>> > Thank you for taking your time.
>> > Heejoon.
>> >
>> >
>> > On Sun, Jun 30, 2013 at 12:24 AM, Chathura Herath
>> > <ch...@cs.indiana.edu>
>> > wrote:
>> >>
>> >> Two questions.
>> >> 1) your services are they public and can i access them.
>> >> 2) Can you send us your xwf file.
>> >>
>> >> If yes to both of the above we can actually load your workflow and
>> >> debug
>> >> this.
>> >>
>> >> If not;
>> >> I looked at the code, the exception you get is because xbaya couldnt
>> >> find the element array. From what i see in the code you need one more
>> >> wrapper element. This is the wrapper element you get when u define in
>> >> the wsdl an element for max occurs unbounded.
>> >>
>> >> I don't want to disappoint you with many trial and errors. you can try
>> >> this if not give us the services and the xwf so one of us can take a
>> >> look.
>> >>
>> >>
>> >> <ComponentOneOrWhateverOutputParamname>
>> >> <somewrapperelement>
>> >> <val>a</val>
>> >> <val>b</val>
>> >> </somewrapperelement>
>> >> ..
>> >> </ComponentOneOrWhateverOutputParamname>
>> >>
>> >> On Sat, Jun 29, 2013 at 3:38 AM, Heejoon Chae <he...@cs.indiana.edu>
>> >> wrote:
>> >> > Hi, Suresh,
>> >> >
>> >> > Any updates on this?
>> >> >
>> >> > By the way, can you also let me know how I cloud run for-each
>> >> > component
>> >> > on
>> >> > files? Actually, what I want to do is to run my script on several
>> >> > input
>> >> > files(let's say 100 inputs), but actual input files' informations are
>> >> > listed
>> >> > on a single file instead of getting all the input one by one.
>> >> >
>> >> > So, the input is single file containing all the input file
>> >> > informations
>> >> > looks like,
>> >> >
>> >> > ...
>> >> > http://xxx,xxx,xxx/input1.txt
>> >> > http://xxx,xxx,xxx/input2.txt
>> >> > http://xxx,xxx,xxx/input3.txt
>> >> > ...
>> >> >
>> >> > Thanks!
>> >> > Heejoon.
>> >> >
>> >> >
>> >> > On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org>
>> >> > wrote:
>> >> >>
>> >> >> Hi Heejoon,
>> >> >>
>> >> >> Normally we use for-each on files, so the way to handle that is let
>> >> >> all
>> >> >> the output files go into a outputData directory and iterate over
>> >> >> them.
>> >> >> But I
>> >> >> see what you are doing here. Let me try this and get back to you
>> >> >> within
>> >> >> the
>> >> >> next day.
>> >> >>
>> >> >> Thanks for your patience,
>> >> >> Suresh
>> >> >>
>> >> >> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu>
>> >> >> wrote:
>> >> >>
>> >> >> Hi, Airavata users,
>> >> >>
>> >> >> Do you have any advises or experiences for my previous mail about
>> >> >> ForEach
>> >> >> component?
>> >> >> I am still struggling with generating input for ForEach component in
>> >> >> my
>> >> >> script :/
>> >> >> Any comment will be helpful to me to use the component :)
>> >> >>
>> >> >> Thank you!
>> >> >>
>> >> >> ---------- Forwarded message ----------
>> >> >> From: Heejoon Chae <he...@cs.indiana.edu>
>> >> >> Date: Fri, Jun 7, 2013 at 5:44 PM
>> >> >> Subject: Re: Question for Airavata For-Each component
>> >> >> To: "users@airavata.apache.org" <us...@airavata.apache.org>
>> >> >>
>> >> >>
>> >> >> Hi, Lahir,
>> >> >>
>> >> >> I succeeded to use ForEach component with attaching 'Input'
>> >> >> component
>> >> >> in
>> >> >> front of ForEach componet with comma separated value(i.e "a,b,c") as
>> >> >> input
>> >> >> like in attached figure. ForEach component works well with Input
>> >> >> component
>> >> >> with comma separate value.
>> >> >>
>> >> >> Now I want to mimic what Input component does within my script as in
>> >> >> another figure. This is required to run ForEach on generated data
>> >> >> not
>> >> >> on
>> >> >> data inputed from user.
>> >> >>
>> >> >> I tired several ways to generate a input for ForEach component in my
>> >> >> script including what you suggested in previous mail, but I
>> >> >> couldn't.
>> >> >> Please
>> >> >> help me to solve this problem.
>> >> >>
>> >> >> Thank you,
>> >> >> Heejoon.
>> >> >>
>> >> >>
>> >> >> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake
>> >> >> <gl...@gmail.com>
>> >> >> wrote:
>> >> >>>
>> >> >>> Following script will be useful for you..
>> >> >>>
>> >> >>> echo $@
>> >> >>> echo $*
>> >> >>> echo $#
>> >> >>> for arg; do
>> >> >>> printf '%s\n' "$arg"
>> >> >>> done
>> >> >>>
>> >> >>>
>> >> >>> Lahir
>> >> >>>
>> >> >>>
>> >> >>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>> >> >>>>
>> >> >>>> Hi Lahiru,
>> >> >>>>
>> >> >>>> Thank you for your answer.
>> >> >>>>
>> >> >>>> By the way, how to create string array in shell script, so that I
>> >> >>>> can
>> >> >>>> pass it to ForEach component?
>> >> >>>>
>> >> >>>> Thank you,
>> >> >>>> Heejoon.
>> >> >>>>
>> >> >>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>> >> >>>>
>> >> >>>> Hi Heejoon,
>> >> >>>>
>> >> >>>> We take string array as input but we invoke the script with each
>> >> >>>> element
>> >> >>>> of the string array multiple times, in Foreach we don't pass whole
>> >> >>>> string
>> >> >>>> array to the application script at once. So you really don't have
>> >> >>>> to
>> >> >>>> handle
>> >> >>>> a string array in your script.
>> >> >>>>
>> >> >>>> Lahiru
>> >> >>>>
>> >> >>>>
>> >> >>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae
>> >> >>>> <he...@cs.indiana.edu>
>> >> >>>> wrote:
>> >> >>>>>
>> >> >>>>> Hi,
>> >> >>>>>
>> >> >>>>> I have quick question about Airavata For-Each component.
>> >> >>>>>
>> >> >>>>> It seems that ForEach component takes String array as an input
>> >> >>>>> and
>> >> >>>>> pass
>> >> >>>>> them to next component one by one.
>> >> >>>>>
>> >> >>>>> My question is how to create string array in shell script to pass
>> >> >>>>> it
>> >> >>>>> to
>> >> >>>>> ForEach component?
>> >> >>>>>
>> >> >>>>> I normally generate string result like
>> >> >>>>>
>> >> >>>>> echo "output=xxx"
>> >> >>>>>
>> >> >>>>> at the end of script and connect it as an input of next component
>> >> >>>>> in
>> >> >>>>> Airavata workflow to pass the result and let next component know
>> >> >>>>> the
>> >> >>>>> input
>> >> >>>>> is arrived.
>> >> >>>>>
>> >> >>>>> Thanks,
>> >> >>>>> Heejoon.
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> --
>> >> >>>> System Analyst Programmer
>> >> >>>> PTI Lab
>> >> >>>> Indiana University
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> System Analyst Programmer
>> >> >>> PTI Lab
>> >> >>> Indiana University
>> >> >>
>> >> >>
>> >> >>
>> >> >> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
>> >> >> 3.11.31 PM.png>
>> >> >>
>> >> >>
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Chathura Herath Ph.D
>> >> http://people.apache.org/~chathura/
>> >> http://chathurah.blogspot.com/
>> >
>> >
>>
>>
>>
>> --
>> Chathura Herath Ph.D
>> http://people.apache.org/~chathura/
>> http://chathurah.blogspot.com/
>
>
--
Chathura Herath Ph.D
http://people.apache.org/~chathura/
http://chathurah.blogspot.com/
Re: Question for Airavata For-Each component
Posted by Heejoon Chae <he...@cs.indiana.edu>.
Hi, Chathura,
For some reason, previous xwf gives errors, so I am re-attaching the
scripts and xwf.
I also attached wsdl from the first component
Thank you!,
Heejoon.
On Sun, Jun 30, 2013 at 2:01 PM, Chathura Herath <ch...@cs.indiana.edu>wrote:
> send me the result xml from your first component as well
>
> On Sat, Jun 29, 2013 at 11:26 PM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> > Hi, Chathura,
> >
> > I am attaching simplified xwf and related scripts.
> >
> > Basically if user input filename which contains list of file names,
> > 'string_array_test.sh' script generates input for 'foreach component'.
> > Then, 'local_echo.sh' will do something on each passed file name.
> >
> > Thank you for taking your time.
> > Heejoon.
> >
> >
> > On Sun, Jun 30, 2013 at 12:24 AM, Chathura Herath <
> cherath@cs.indiana.edu>
> > wrote:
> >>
> >> Two questions.
> >> 1) your services are they public and can i access them.
> >> 2) Can you send us your xwf file.
> >>
> >> If yes to both of the above we can actually load your workflow and debug
> >> this.
> >>
> >> If not;
> >> I looked at the code, the exception you get is because xbaya couldnt
> >> find the element array. From what i see in the code you need one more
> >> wrapper element. This is the wrapper element you get when u define in
> >> the wsdl an element for max occurs unbounded.
> >>
> >> I don't want to disappoint you with many trial and errors. you can try
> >> this if not give us the services and the xwf so one of us can take a
> >> look.
> >>
> >>
> >> <ComponentOneOrWhateverOutputParamname>
> >> <somewrapperelement>
> >> <val>a</val>
> >> <val>b</val>
> >> </somewrapperelement>
> >> ..
> >> </ComponentOneOrWhateverOutputParamname>
> >>
> >> On Sat, Jun 29, 2013 at 3:38 AM, Heejoon Chae <he...@cs.indiana.edu>
> >> wrote:
> >> > Hi, Suresh,
> >> >
> >> > Any updates on this?
> >> >
> >> > By the way, can you also let me know how I cloud run for-each
> component
> >> > on
> >> > files? Actually, what I want to do is to run my script on several
> input
> >> > files(let's say 100 inputs), but actual input files' informations are
> >> > listed
> >> > on a single file instead of getting all the input one by one.
> >> >
> >> > So, the input is single file containing all the input file
> informations
> >> > looks like,
> >> >
> >> > ...
> >> > http://xxx,xxx,xxx/input1.txt
> >> > http://xxx,xxx,xxx/input2.txt
> >> > http://xxx,xxx,xxx/input3.txt
> >> > ...
> >> >
> >> > Thanks!
> >> > Heejoon.
> >> >
> >> >
> >> > On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org>
> >> > wrote:
> >> >>
> >> >> Hi Heejoon,
> >> >>
> >> >> Normally we use for-each on files, so the way to handle that is let
> all
> >> >> the output files go into a outputData directory and iterate over
> them.
> >> >> But I
> >> >> see what you are doing here. Let me try this and get back to you
> within
> >> >> the
> >> >> next day.
> >> >>
> >> >> Thanks for your patience,
> >> >> Suresh
> >> >>
> >> >> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu>
> >> >> wrote:
> >> >>
> >> >> Hi, Airavata users,
> >> >>
> >> >> Do you have any advises or experiences for my previous mail about
> >> >> ForEach
> >> >> component?
> >> >> I am still struggling with generating input for ForEach component in
> my
> >> >> script :/
> >> >> Any comment will be helpful to me to use the component :)
> >> >>
> >> >> Thank you!
> >> >>
> >> >> ---------- Forwarded message ----------
> >> >> From: Heejoon Chae <he...@cs.indiana.edu>
> >> >> Date: Fri, Jun 7, 2013 at 5:44 PM
> >> >> Subject: Re: Question for Airavata For-Each component
> >> >> To: "users@airavata.apache.org" <us...@airavata.apache.org>
> >> >>
> >> >>
> >> >> Hi, Lahir,
> >> >>
> >> >> I succeeded to use ForEach component with attaching 'Input'
> component
> >> >> in
> >> >> front of ForEach componet with comma separated value(i.e "a,b,c") as
> >> >> input
> >> >> like in attached figure. ForEach component works well with Input
> >> >> component
> >> >> with comma separate value.
> >> >>
> >> >> Now I want to mimic what Input component does within my script as in
> >> >> another figure. This is required to run ForEach on generated data not
> >> >> on
> >> >> data inputed from user.
> >> >>
> >> >> I tired several ways to generate a input for ForEach component in my
> >> >> script including what you suggested in previous mail, but I couldn't.
> >> >> Please
> >> >> help me to solve this problem.
> >> >>
> >> >> Thank you,
> >> >> Heejoon.
> >> >>
> >> >>
> >> >> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <
> glahiru@gmail.com>
> >> >> wrote:
> >> >>>
> >> >>> Following script will be useful for you..
> >> >>>
> >> >>> echo $@
> >> >>> echo $*
> >> >>> echo $#
> >> >>> for arg; do
> >> >>> printf '%s\n' "$arg"
> >> >>> done
> >> >>>
> >> >>>
> >> >>> Lahir
> >> >>>
> >> >>>
> >> >>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
> >> >>>>
> >> >>>> Hi Lahiru,
> >> >>>>
> >> >>>> Thank you for your answer.
> >> >>>>
> >> >>>> By the way, how to create string array in shell script, so that I
> can
> >> >>>> pass it to ForEach component?
> >> >>>>
> >> >>>> Thank you,
> >> >>>> Heejoon.
> >> >>>>
> >> >>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
> >> >>>>
> >> >>>> Hi Heejoon,
> >> >>>>
> >> >>>> We take string array as input but we invoke the script with each
> >> >>>> element
> >> >>>> of the string array multiple times, in Foreach we don't pass whole
> >> >>>> string
> >> >>>> array to the application script at once. So you really don't have
> to
> >> >>>> handle
> >> >>>> a string array in your script.
> >> >>>>
> >> >>>> Lahiru
> >> >>>>
> >> >>>>
> >> >>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae
> >> >>>> <he...@cs.indiana.edu>
> >> >>>> wrote:
> >> >>>>>
> >> >>>>> Hi,
> >> >>>>>
> >> >>>>> I have quick question about Airavata For-Each component.
> >> >>>>>
> >> >>>>> It seems that ForEach component takes String array as an input and
> >> >>>>> pass
> >> >>>>> them to next component one by one.
> >> >>>>>
> >> >>>>> My question is how to create string array in shell script to pass
> it
> >> >>>>> to
> >> >>>>> ForEach component?
> >> >>>>>
> >> >>>>> I normally generate string result like
> >> >>>>>
> >> >>>>> echo "output=xxx"
> >> >>>>>
> >> >>>>> at the end of script and connect it as an input of next component
> in
> >> >>>>> Airavata workflow to pass the result and let next component know
> the
> >> >>>>> input
> >> >>>>> is arrived.
> >> >>>>>
> >> >>>>> Thanks,
> >> >>>>> Heejoon.
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> --
> >> >>>> System Analyst Programmer
> >> >>>> PTI Lab
> >> >>>> Indiana University
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> System Analyst Programmer
> >> >>> PTI Lab
> >> >>> Indiana University
> >> >>
> >> >>
> >> >>
> >> >> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
> >> >> 3.11.31 PM.png>
> >> >>
> >> >>
> >> >
> >>
> >>
> >>
> >> --
> >> Chathura Herath Ph.D
> >> http://people.apache.org/~chathura/
> >> http://chathurah.blogspot.com/
> >
> >
>
>
>
> --
> Chathura Herath Ph.D
> http://people.apache.org/~chathura/
> http://chathurah.blogspot.com/
>
Re: Question for Airavata For-Each component
Posted by Chathura Herath <ch...@cs.indiana.edu>.
send me the result xml from your first component as well
On Sat, Jun 29, 2013 at 11:26 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
> Hi, Chathura,
>
> I am attaching simplified xwf and related scripts.
>
> Basically if user input filename which contains list of file names,
> 'string_array_test.sh' script generates input for 'foreach component'.
> Then, 'local_echo.sh' will do something on each passed file name.
>
> Thank you for taking your time.
> Heejoon.
>
>
> On Sun, Jun 30, 2013 at 12:24 AM, Chathura Herath <ch...@cs.indiana.edu>
> wrote:
>>
>> Two questions.
>> 1) your services are they public and can i access them.
>> 2) Can you send us your xwf file.
>>
>> If yes to both of the above we can actually load your workflow and debug
>> this.
>>
>> If not;
>> I looked at the code, the exception you get is because xbaya couldnt
>> find the element array. From what i see in the code you need one more
>> wrapper element. This is the wrapper element you get when u define in
>> the wsdl an element for max occurs unbounded.
>>
>> I don't want to disappoint you with many trial and errors. you can try
>> this if not give us the services and the xwf so one of us can take a
>> look.
>>
>>
>> <ComponentOneOrWhateverOutputParamname>
>> <somewrapperelement>
>> <val>a</val>
>> <val>b</val>
>> </somewrapperelement>
>> ..
>> </ComponentOneOrWhateverOutputParamname>
>>
>> On Sat, Jun 29, 2013 at 3:38 AM, Heejoon Chae <he...@cs.indiana.edu>
>> wrote:
>> > Hi, Suresh,
>> >
>> > Any updates on this?
>> >
>> > By the way, can you also let me know how I cloud run for-each component
>> > on
>> > files? Actually, what I want to do is to run my script on several input
>> > files(let's say 100 inputs), but actual input files' informations are
>> > listed
>> > on a single file instead of getting all the input one by one.
>> >
>> > So, the input is single file containing all the input file informations
>> > looks like,
>> >
>> > ...
>> > http://xxx,xxx,xxx/input1.txt
>> > http://xxx,xxx,xxx/input2.txt
>> > http://xxx,xxx,xxx/input3.txt
>> > ...
>> >
>> > Thanks!
>> > Heejoon.
>> >
>> >
>> > On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org>
>> > wrote:
>> >>
>> >> Hi Heejoon,
>> >>
>> >> Normally we use for-each on files, so the way to handle that is let all
>> >> the output files go into a outputData directory and iterate over them.
>> >> But I
>> >> see what you are doing here. Let me try this and get back to you within
>> >> the
>> >> next day.
>> >>
>> >> Thanks for your patience,
>> >> Suresh
>> >>
>> >> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu>
>> >> wrote:
>> >>
>> >> Hi, Airavata users,
>> >>
>> >> Do you have any advises or experiences for my previous mail about
>> >> ForEach
>> >> component?
>> >> I am still struggling with generating input for ForEach component in my
>> >> script :/
>> >> Any comment will be helpful to me to use the component :)
>> >>
>> >> Thank you!
>> >>
>> >> ---------- Forwarded message ----------
>> >> From: Heejoon Chae <he...@cs.indiana.edu>
>> >> Date: Fri, Jun 7, 2013 at 5:44 PM
>> >> Subject: Re: Question for Airavata For-Each component
>> >> To: "users@airavata.apache.org" <us...@airavata.apache.org>
>> >>
>> >>
>> >> Hi, Lahir,
>> >>
>> >> I succeeded to use ForEach component with attaching 'Input' component
>> >> in
>> >> front of ForEach componet with comma separated value(i.e "a,b,c") as
>> >> input
>> >> like in attached figure. ForEach component works well with Input
>> >> component
>> >> with comma separate value.
>> >>
>> >> Now I want to mimic what Input component does within my script as in
>> >> another figure. This is required to run ForEach on generated data not
>> >> on
>> >> data inputed from user.
>> >>
>> >> I tired several ways to generate a input for ForEach component in my
>> >> script including what you suggested in previous mail, but I couldn't.
>> >> Please
>> >> help me to solve this problem.
>> >>
>> >> Thank you,
>> >> Heejoon.
>> >>
>> >>
>> >> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Following script will be useful for you..
>> >>>
>> >>> echo $@
>> >>> echo $*
>> >>> echo $#
>> >>> for arg; do
>> >>> printf '%s\n' "$arg"
>> >>> done
>> >>>
>> >>>
>> >>> Lahir
>> >>>
>> >>>
>> >>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>> >>>>
>> >>>> Hi Lahiru,
>> >>>>
>> >>>> Thank you for your answer.
>> >>>>
>> >>>> By the way, how to create string array in shell script, so that I can
>> >>>> pass it to ForEach component?
>> >>>>
>> >>>> Thank you,
>> >>>> Heejoon.
>> >>>>
>> >>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>> >>>>
>> >>>> Hi Heejoon,
>> >>>>
>> >>>> We take string array as input but we invoke the script with each
>> >>>> element
>> >>>> of the string array multiple times, in Foreach we don't pass whole
>> >>>> string
>> >>>> array to the application script at once. So you really don't have to
>> >>>> handle
>> >>>> a string array in your script.
>> >>>>
>> >>>> Lahiru
>> >>>>
>> >>>>
>> >>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae
>> >>>> <he...@cs.indiana.edu>
>> >>>> wrote:
>> >>>>>
>> >>>>> Hi,
>> >>>>>
>> >>>>> I have quick question about Airavata For-Each component.
>> >>>>>
>> >>>>> It seems that ForEach component takes String array as an input and
>> >>>>> pass
>> >>>>> them to next component one by one.
>> >>>>>
>> >>>>> My question is how to create string array in shell script to pass it
>> >>>>> to
>> >>>>> ForEach component?
>> >>>>>
>> >>>>> I normally generate string result like
>> >>>>>
>> >>>>> echo "output=xxx"
>> >>>>>
>> >>>>> at the end of script and connect it as an input of next component in
>> >>>>> Airavata workflow to pass the result and let next component know the
>> >>>>> input
>> >>>>> is arrived.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Heejoon.
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> System Analyst Programmer
>> >>>> PTI Lab
>> >>>> Indiana University
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> System Analyst Programmer
>> >>> PTI Lab
>> >>> Indiana University
>> >>
>> >>
>> >>
>> >> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
>> >> 3.11.31 PM.png>
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Chathura Herath Ph.D
>> http://people.apache.org/~chathura/
>> http://chathurah.blogspot.com/
>
>
--
Chathura Herath Ph.D
http://people.apache.org/~chathura/
http://chathurah.blogspot.com/
Re: Question for Airavata For-Each component
Posted by Heejoon Chae <he...@cs.indiana.edu>.
Hi, Chathura,
I am attaching simplified xwf and related scripts.
Basically if user input filename which contains list of file names,
'string_array_test.sh' script generates input for 'foreach component'.
Then, 'local_echo.sh' will do something on each passed file name.
Thank you for taking your time.
Heejoon.
On Sun, Jun 30, 2013 at 12:24 AM, Chathura Herath <ch...@cs.indiana.edu>wrote:
> Two questions.
> 1) your services are they public and can i access them.
> 2) Can you send us your xwf file.
>
> If yes to both of the above we can actually load your workflow and debug
> this.
>
> If not;
> I looked at the code, the exception you get is because xbaya couldnt
> find the element array. From what i see in the code you need one more
> wrapper element. This is the wrapper element you get when u define in
> the wsdl an element for max occurs unbounded.
>
> I don't want to disappoint you with many trial and errors. you can try
> this if not give us the services and the xwf so one of us can take a
> look.
>
>
> <ComponentOneOrWhateverOutputParamname>
> <somewrapperelement>
> <val>a</val>
> <val>b</val>
> </somewrapperelement>
> ..
> </ComponentOneOrWhateverOutputParamname>
>
> On Sat, Jun 29, 2013 at 3:38 AM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> > Hi, Suresh,
> >
> > Any updates on this?
> >
> > By the way, can you also let me know how I cloud run for-each component
> on
> > files? Actually, what I want to do is to run my script on several input
> > files(let's say 100 inputs), but actual input files' informations are
> listed
> > on a single file instead of getting all the input one by one.
> >
> > So, the input is single file containing all the input file informations
> > looks like,
> >
> > ...
> > http://xxx,xxx,xxx/input1.txt
> > http://xxx,xxx,xxx/input2.txt
> > http://xxx,xxx,xxx/input3.txt
> > ...
> >
> > Thanks!
> > Heejoon.
> >
> >
> > On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org>
> wrote:
> >>
> >> Hi Heejoon,
> >>
> >> Normally we use for-each on files, so the way to handle that is let all
> >> the output files go into a outputData directory and iterate over them.
> But I
> >> see what you are doing here. Let me try this and get back to you within
> the
> >> next day.
> >>
> >> Thanks for your patience,
> >> Suresh
> >>
> >> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> >>
> >> Hi, Airavata users,
> >>
> >> Do you have any advises or experiences for my previous mail about
> ForEach
> >> component?
> >> I am still struggling with generating input for ForEach component in my
> >> script :/
> >> Any comment will be helpful to me to use the component :)
> >>
> >> Thank you!
> >>
> >> ---------- Forwarded message ----------
> >> From: Heejoon Chae <he...@cs.indiana.edu>
> >> Date: Fri, Jun 7, 2013 at 5:44 PM
> >> Subject: Re: Question for Airavata For-Each component
> >> To: "users@airavata.apache.org" <us...@airavata.apache.org>
> >>
> >>
> >> Hi, Lahir,
> >>
> >> I succeeded to use ForEach component with attaching 'Input' component
> in
> >> front of ForEach componet with comma separated value(i.e "a,b,c") as
> input
> >> like in attached figure. ForEach component works well with Input
> component
> >> with comma separate value.
> >>
> >> Now I want to mimic what Input component does within my script as in
> >> another figure. This is required to run ForEach on generated data not on
> >> data inputed from user.
> >>
> >> I tired several ways to generate a input for ForEach component in my
> >> script including what you suggested in previous mail, but I couldn't.
> Please
> >> help me to solve this problem.
> >>
> >> Thank you,
> >> Heejoon.
> >>
> >>
> >> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>
> >> wrote:
> >>>
> >>> Following script will be useful for you..
> >>>
> >>> echo $@
> >>> echo $*
> >>> echo $#
> >>> for arg; do
> >>> printf '%s\n' "$arg"
> >>> done
> >>>
> >>>
> >>> Lahir
> >>>
> >>>
> >>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
> >>>>
> >>>> Hi Lahiru,
> >>>>
> >>>> Thank you for your answer.
> >>>>
> >>>> By the way, how to create string array in shell script, so that I can
> >>>> pass it to ForEach component?
> >>>>
> >>>> Thank you,
> >>>> Heejoon.
> >>>>
> >>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
> >>>>
> >>>> Hi Heejoon,
> >>>>
> >>>> We take string array as input but we invoke the script with each
> element
> >>>> of the string array multiple times, in Foreach we don't pass whole
> string
> >>>> array to the application script at once. So you really don't have to
> handle
> >>>> a string array in your script.
> >>>>
> >>>> Lahiru
> >>>>
> >>>>
> >>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <heechae@cs.indiana.edu
> >
> >>>> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I have quick question about Airavata For-Each component.
> >>>>>
> >>>>> It seems that ForEach component takes String array as an input and
> pass
> >>>>> them to next component one by one.
> >>>>>
> >>>>> My question is how to create string array in shell script to pass it
> to
> >>>>> ForEach component?
> >>>>>
> >>>>> I normally generate string result like
> >>>>>
> >>>>> echo "output=xxx"
> >>>>>
> >>>>> at the end of script and connect it as an input of next component in
> >>>>> Airavata workflow to pass the result and let next component know the
> input
> >>>>> is arrived.
> >>>>>
> >>>>> Thanks,
> >>>>> Heejoon.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> System Analyst Programmer
> >>>> PTI Lab
> >>>> Indiana University
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> System Analyst Programmer
> >>> PTI Lab
> >>> Indiana University
> >>
> >>
> >>
> >> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
> >> 3.11.31 PM.png>
> >>
> >>
> >
>
>
>
> --
> Chathura Herath Ph.D
> http://people.apache.org/~chathura/
> http://chathurah.blogspot.com/
>
Re: Question for Airavata For-Each component
Posted by Chathura Herath <ch...@cs.indiana.edu>.
Two questions.
1) your services are they public and can i access them.
2) Can you send us your xwf file.
If yes to both of the above we can actually load your workflow and debug this.
If not;
I looked at the code, the exception you get is because xbaya couldnt
find the element array. From what i see in the code you need one more
wrapper element. This is the wrapper element you get when u define in
the wsdl an element for max occurs unbounded.
I don't want to disappoint you with many trial and errors. you can try
this if not give us the services and the xwf so one of us can take a
look.
<ComponentOneOrWhateverOutputParamname>
<somewrapperelement>
<val>a</val>
<val>b</val>
</somewrapperelement>
..
</ComponentOneOrWhateverOutputParamname>
On Sat, Jun 29, 2013 at 3:38 AM, Heejoon Chae <he...@cs.indiana.edu> wrote:
> Hi, Suresh,
>
> Any updates on this?
>
> By the way, can you also let me know how I cloud run for-each component on
> files? Actually, what I want to do is to run my script on several input
> files(let's say 100 inputs), but actual input files' informations are listed
> on a single file instead of getting all the input one by one.
>
> So, the input is single file containing all the input file informations
> looks like,
>
> ...
> http://xxx,xxx,xxx/input1.txt
> http://xxx,xxx,xxx/input2.txt
> http://xxx,xxx,xxx/input3.txt
> ...
>
> Thanks!
> Heejoon.
>
>
> On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org> wrote:
>>
>> Hi Heejoon,
>>
>> Normally we use for-each on files, so the way to handle that is let all
>> the output files go into a outputData directory and iterate over them. But I
>> see what you are doing here. Let me try this and get back to you within the
>> next day.
>>
>> Thanks for your patience,
>> Suresh
>>
>> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>>
>> Hi, Airavata users,
>>
>> Do you have any advises or experiences for my previous mail about ForEach
>> component?
>> I am still struggling with generating input for ForEach component in my
>> script :/
>> Any comment will be helpful to me to use the component :)
>>
>> Thank you!
>>
>> ---------- Forwarded message ----------
>> From: Heejoon Chae <he...@cs.indiana.edu>
>> Date: Fri, Jun 7, 2013 at 5:44 PM
>> Subject: Re: Question for Airavata For-Each component
>> To: "users@airavata.apache.org" <us...@airavata.apache.org>
>>
>>
>> Hi, Lahir,
>>
>> I succeeded to use ForEach component with attaching 'Input' component in
>> front of ForEach componet with comma separated value(i.e "a,b,c") as input
>> like in attached figure. ForEach component works well with Input component
>> with comma separate value.
>>
>> Now I want to mimic what Input component does within my script as in
>> another figure. This is required to run ForEach on generated data not on
>> data inputed from user.
>>
>> I tired several ways to generate a input for ForEach component in my
>> script including what you suggested in previous mail, but I couldn't. Please
>> help me to solve this problem.
>>
>> Thank you,
>> Heejoon.
>>
>>
>> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>
>> wrote:
>>>
>>> Following script will be useful for you..
>>>
>>> echo $@
>>> echo $*
>>> echo $#
>>> for arg; do
>>> printf '%s\n' "$arg"
>>> done
>>>
>>>
>>> Lahir
>>>
>>>
>>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>>>>
>>>> Hi Lahiru,
>>>>
>>>> Thank you for your answer.
>>>>
>>>> By the way, how to create string array in shell script, so that I can
>>>> pass it to ForEach component?
>>>>
>>>> Thank you,
>>>> Heejoon.
>>>>
>>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>>>>
>>>> Hi Heejoon,
>>>>
>>>> We take string array as input but we invoke the script with each element
>>>> of the string array multiple times, in Foreach we don't pass whole string
>>>> array to the application script at once. So you really don't have to handle
>>>> a string array in your script.
>>>>
>>>> Lahiru
>>>>
>>>>
>>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I have quick question about Airavata For-Each component.
>>>>>
>>>>> It seems that ForEach component takes String array as an input and pass
>>>>> them to next component one by one.
>>>>>
>>>>> My question is how to create string array in shell script to pass it to
>>>>> ForEach component?
>>>>>
>>>>> I normally generate string result like
>>>>>
>>>>> echo "output=xxx"
>>>>>
>>>>> at the end of script and connect it as an input of next component in
>>>>> Airavata workflow to pass the result and let next component know the input
>>>>> is arrived.
>>>>>
>>>>> Thanks,
>>>>> Heejoon.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> System Analyst Programmer
>>>> PTI Lab
>>>> Indiana University
>>>
>>>
>>>
>>>
>>> --
>>> System Analyst Programmer
>>> PTI Lab
>>> Indiana University
>>
>>
>>
>> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
>> 3.11.31 PM.png>
>>
>>
>
--
Chathura Herath Ph.D
http://people.apache.org/~chathura/
http://chathurah.blogspot.com/
Re: Question for Airavata For-Each component
Posted by Heejoon Chae <he...@cs.indiana.edu>.
Hi, Suresh,
On Tue, Jul 2, 2013 at 11:47 AM, Suresh Marru <sm...@apache.org> wrote:
> On Jun 29, 2013, at 3:38 AM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>
> > Hi, Suresh,
> >
> > Any updates on this?
> >
> > By the way, can you also let me know how I cloud run for-each component
> on files? Actually, what I want to do is to run my script on several input
> files(let's say 100 inputs), but actual input files' informations are
> listed on a single file instead of getting all the input one by one.
> >
> > So, the input is single file containing all the input file informations
> looks like,
> >
> > ...
> > http://xxx,xxx,xxx/input1.txt
> > http://xxx,xxx,xxx/input2.txt
> > http://xxx,xxx,xxx/input3.txt
> > …
>
> Hi Heejoon,
>
> While Chathura and Saminda help you get pass the for-each issues, let me
> answer this specific question. I have seen a similar requirement previously
> in a different project [1] as well. It should be trivial to add these kind
> of "convenience" nodes in workflow which can read a file and split the
> content and pass to for-each as an array. We can develop these nodes, but
> can you please spare some time and formalize this requirement.
>
> * Other than the ones you are scripting, did you come across similar
> bioinformatics applications or otherwise which produce such listing? Did
> you see and ad-hoc standards for these formats? Like what is the delimiter
> for each file, newline? it can be comma or space separated as well right?
> do you think if Airavata establishes some norm, will application writers
> follow them?
>
What I want to handle here with for each component is input data and I
guess one of the key difference between bioinformatics and others are
number of data and their size.
Let's say I have 30 biological data inputs which is normall in this area. I
guess it could be better way to handle them, but simple solution that I
imagine is listing & parsing. Without the listing and parsing the inputs, I
need to drag and drop 30 input components which will mess up the workflow
by only inputs. As you mentioned space separated, tab or comma separated, I
guess they are totally fine with couple of inputs, but you could
simply imagine long long 30 inputs with containing delimiters.
> * How do we determine the type of the parsed content? For instance,
> distinguishing form a URI, or a String or integer?
>
In the example, the reason I used URI is that I assumes the system will be
used by users with their own data. As I mentioned the data size in
bioinformatics is normally large(for example 1 cell line per 5GB) and they
are deposited as web accessible locations. My script download them using
parsed URIs and start analysis. So, It might be useful if there is a system
component which will handle several inputs and large data at once. In
general, bioinformatics applications generate text file as output, but the
type(or format) is varies which is one of the issues in this area as well.
Thanks,
Heejoon.
>
> I am sure there will be other questions, but once we agree upon such
> semantic issues of the content, we can add this as a feature request for
> next release.
>
> Suresh
>
> [1] - https://cwiki.apache.org/confluence/display/AIRAVATA/ODI
>
> >
> > Thanks!
> > Heejoon.
> >
> >
> > On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org>
> wrote:
> > Hi Heejoon,
> >
> > Normally we use for-each on files, so the way to handle that is let all
> the output files go into a outputData directory and iterate over them. But
> I see what you are doing here. Let me try this and get back to you within
> the next day.
> >
> > Thanks for your patience,
> > Suresh
> >
> > On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> >
> >> Hi, Airavata users,
> >>
> >> Do you have any advises or experiences for my previous mail about
> ForEach component?
> >> I am still struggling with generating input for ForEach component in my
> script :/
> >> Any comment will be helpful to me to use the component :)
> >>
> >> Thank you!
> >>
> >> ---------- Forwarded message ----------
> >> From: Heejoon Chae <he...@cs.indiana.edu>
> >> Date: Fri, Jun 7, 2013 at 5:44 PM
> >> Subject: Re: Question for Airavata For-Each component
> >> To: "users@airavata.apache.org" <us...@airavata.apache.org>
> >>
> >>
> >> Hi, Lahir,
> >>
> >> I succeeded to use ForEach component with attaching 'Input' component
> in front of ForEach componet with comma separated value(i.e "a,b,c") as
> input like in attached figure. ForEach component works well with Input
> component with comma separate value.
> >>
> >> Now I want to mimic what Input component does within my script as in
> another figure. This is required to run ForEach on generated data not on
> data inputed from user.
> >>
> >> I tired several ways to generate a input for ForEach component in my
> script including what you suggested in previous mail, but I couldn't.
> Please help me to solve this problem.
> >>
> >> Thank you,
> >> Heejoon.
> >>
> >>
> >> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>
> wrote:
> >> Following script will be useful for you..
> >>
> >> echo $@
> >> echo $*
> >> echo $#
> >> for arg; do
> >> printf '%s\n' "$arg"
> >> done
> >>
> >>
> >> Lahir
> >>
> >>
> >> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
> >> Hi Lahiru,
> >>
> >> Thank you for your answer.
> >>
> >> By the way, how to create string array in shell script, so that I can
> pass it to ForEach component?
> >>
> >> Thank you,
> >> Heejoon.
> >>
> >> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
> >>
> >>> Hi Heejoon,
> >>>
> >>> We take string array as input but we invoke the script with each
> element of the string array multiple times, in Foreach we don't pass whole
> string array to the application script at once. So you really don't have to
> handle a string array in your script.
> >>>
> >>> Lahiru
> >>>
> >>>
> >>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>
> wrote:
> >>> Hi,
> >>>
> >>> I have quick question about Airavata For-Each component.
> >>>
> >>> It seems that ForEach component takes String array as an input and
> pass them to next component one by one.
> >>>
> >>> My question is how to create string array in shell script to pass it
> to ForEach component?
> >>>
> >>> I normally generate string result like
> >>>
> >>> echo "output=xxx"
> >>>
> >>> at the end of script and connect it as an input of next component in
> Airavata workflow to pass the result and let next component know the input
> is arrived.
> >>>
> >>> Thanks,
> >>> Heejoon.
> >>>
> >>>
> >>>
> >>> --
> >>> System Analyst Programmer
> >>> PTI Lab
> >>> Indiana University
> >>
> >>
> >>
> >> --
> >> System Analyst Programmer
> >> PTI Lab
> >> Indiana University
> >>
> >>
> >> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
> 3.11.31 PM.png>
> >
> >
>
>
Re: Question for Airavata For-Each component
Posted by Suresh Marru <sm...@apache.org>.
On Jun 29, 2013, at 3:38 AM, Heejoon Chae <he...@cs.indiana.edu> wrote:
> Hi, Suresh,
>
> Any updates on this?
>
> By the way, can you also let me know how I cloud run for-each component on files? Actually, what I want to do is to run my script on several input files(let's say 100 inputs), but actual input files' informations are listed on a single file instead of getting all the input one by one.
>
> So, the input is single file containing all the input file informations looks like,
>
> ...
> http://xxx,xxx,xxx/input1.txt
> http://xxx,xxx,xxx/input2.txt
> http://xxx,xxx,xxx/input3.txt
> …
Hi Heejoon,
While Chathura and Saminda help you get pass the for-each issues, let me answer this specific question. I have seen a similar requirement previously in a different project [1] as well. It should be trivial to add these kind of "convenience" nodes in workflow which can read a file and split the content and pass to for-each as an array. We can develop these nodes, but can you please spare some time and formalize this requirement.
* Other than the ones you are scripting, did you come across similar bioinformatics applications or otherwise which produce such listing? Did you see and ad-hoc standards for these formats? Like what is the delimiter for each file, newline? it can be comma or space separated as well right? do you think if Airavata establishes some norm, will application writers follow them?
* How do we determine the type of the parsed content? For instance, distinguishing form a URI, or a String or integer?
I am sure there will be other questions, but once we agree upon such semantic issues of the content, we can add this as a feature request for next release.
Suresh
[1] - https://cwiki.apache.org/confluence/display/AIRAVATA/ODI
>
> Thanks!
> Heejoon.
>
>
> On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org> wrote:
> Hi Heejoon,
>
> Normally we use for-each on files, so the way to handle that is let all the output files go into a outputData directory and iterate over them. But I see what you are doing here. Let me try this and get back to you within the next day.
>
> Thanks for your patience,
> Suresh
>
> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>
>> Hi, Airavata users,
>>
>> Do you have any advises or experiences for my previous mail about ForEach component?
>> I am still struggling with generating input for ForEach component in my script :/
>> Any comment will be helpful to me to use the component :)
>>
>> Thank you!
>>
>> ---------- Forwarded message ----------
>> From: Heejoon Chae <he...@cs.indiana.edu>
>> Date: Fri, Jun 7, 2013 at 5:44 PM
>> Subject: Re: Question for Airavata For-Each component
>> To: "users@airavata.apache.org" <us...@airavata.apache.org>
>>
>>
>> Hi, Lahir,
>>
>> I succeeded to use ForEach component with attaching 'Input' component in front of ForEach componet with comma separated value(i.e "a,b,c") as input like in attached figure. ForEach component works well with Input component with comma separate value.
>>
>> Now I want to mimic what Input component does within my script as in another figure. This is required to run ForEach on generated data not on data inputed from user.
>>
>> I tired several ways to generate a input for ForEach component in my script including what you suggested in previous mail, but I couldn't. Please help me to solve this problem.
>>
>> Thank you,
>> Heejoon.
>>
>>
>> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com> wrote:
>> Following script will be useful for you..
>>
>> echo $@
>> echo $*
>> echo $#
>> for arg; do
>> printf '%s\n' "$arg"
>> done
>>
>>
>> Lahir
>>
>>
>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>> Hi Lahiru,
>>
>> Thank you for your answer.
>>
>> By the way, how to create string array in shell script, so that I can pass it to ForEach component?
>>
>> Thank you,
>> Heejoon.
>>
>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>>
>>> Hi Heejoon,
>>>
>>> We take string array as input but we invoke the script with each element of the string array multiple times, in Foreach we don't pass whole string array to the application script at once. So you really don't have to handle a string array in your script.
>>>
>>> Lahiru
>>>
>>>
>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>>> Hi,
>>>
>>> I have quick question about Airavata For-Each component.
>>>
>>> It seems that ForEach component takes String array as an input and pass them to next component one by one.
>>>
>>> My question is how to create string array in shell script to pass it to ForEach component?
>>>
>>> I normally generate string result like
>>>
>>> echo "output=xxx"
>>>
>>> at the end of script and connect it as an input of next component in Airavata workflow to pass the result and let next component know the input is arrived.
>>>
>>> Thanks,
>>> Heejoon.
>>>
>>>
>>>
>>> --
>>> System Analyst Programmer
>>> PTI Lab
>>> Indiana University
>>
>>
>>
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University
>>
>>
>> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at 3.11.31 PM.png>
>
>
Re: Question for Airavata For-Each component
Posted by Heejoon Chae <he...@cs.indiana.edu>.
Hi, Suresh,
Any updates on this?
By the way, can you also let me know how I cloud run for-each component on
files? Actually, what I want to do is to run my script on several input
files(let's say 100 inputs), but actual input files' informations are
listed on a single file instead of getting all the input one by one.
So, the input is single file containing all the input file informations
looks like,
...
http://xxx,xxx,xxx/input1.txt
http://xxx,xxx,xxx/input2.txt
http://xxx,xxx,xxx/input3.txt
...
Thanks!
Heejoon.
On Tue, Jun 11, 2013 at 10:26 PM, Suresh Marru <sm...@apache.org> wrote:
> Hi Heejoon,
>
> Normally we use for-each on files, so the way to handle that is let all
> the output files go into a outputData directory and iterate over them. But
> I see what you are doing here. Let me try this and get back to you within
> the next day.
>
> Thanks for your patience,
> Suresh
>
> On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>
> Hi, Airavata users,
>
> Do you have any advises or experiences for my previous mail about ForEach
> component?
> I am still struggling with generating input for ForEach component in my
> script :/
> Any comment will be helpful to me to use the component :)
>
> Thank you!
>
> ---------- Forwarded message ----------
> From: Heejoon Chae <he...@cs.indiana.edu>
> Date: Fri, Jun 7, 2013 at 5:44 PM
> Subject: Re: Question for Airavata For-Each component
> To: "users@airavata.apache.org" <us...@airavata.apache.org>
>
>
> Hi, Lahir,
>
> I succeeded to use ForEach component with attaching 'Input' component in
> front of ForEach componet with comma separated value(i.e "a,b,c") as input
> like in attached figure. ForEach component works well with Input component
> with comma separate value.
>
> Now I want to mimic what Input component does within my script as in
> another figure. This is required to run ForEach on generated data not on
> data inputed from user.
>
> I tired several ways to generate a input for ForEach component in my
> script including what you suggested in previous mail, but I couldn't.
> Please help me to solve this problem.
>
> Thank you,
> Heejoon.
>
>
> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>wrote:
>
>> Following script will be useful for you..
>>
>> echo $@
>> echo $*
>> echo $#
>> for arg; do
>> printf '%s\n' "$arg"
>> done
>>
>>
>> Lahir
>>
>>
>> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>>
>>> Hi Lahiru,
>>>
>>> Thank you for your answer.
>>>
>>> By the way, how to create string array in shell script, so that I can
>>> pass it to ForEach component?
>>>
>>> Thank you,
>>> Heejoon.
>>>
>>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>>>
>>> Hi Heejoon,
>>>
>>> We take string array as input but we invoke the script with each element
>>> of the string array multiple times, in Foreach we don't pass whole string
>>> array to the application script at once. So you really don't have to handle
>>> a string array in your script.
>>>
>>> Lahiru
>>>
>>>
>>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>wrote:
>>>
>>>> Hi,
>>>>
>>>> I have quick question about Airavata For-Each component.
>>>>
>>>> It seems that ForEach component takes String array as an input and pass
>>>> them to next component one by one.
>>>>
>>>> My question is how to create string array in shell script to pass it to
>>>> ForEach component?
>>>>
>>>> I normally generate string result like
>>>>
>>>> echo "output=xxx"
>>>>
>>>> at the end of script and connect it as an input of next component in
>>>> Airavata workflow to pass the result and let next component know the input
>>>> is arrived.
>>>>
>>>> Thanks,
>>>> Heejoon.
>>>>
>>>
>>>
>>>
>>> --
>>> System Analyst Programmer
>>> PTI Lab
>>> Indiana University
>>>
>>>
>>
>>
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University
>>
>
>
> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at
> 3.11.31 PM.png>
>
>
>
Re: Question for Airavata For-Each component
Posted by Suresh Marru <sm...@apache.org>.
Hi Heejoon,
Normally we use for-each on files, so the way to handle that is let all the output files go into a outputData directory and iterate over them. But I see what you are doing here. Let me try this and get back to you within the next day.
Thanks for your patience,
Suresh
On Jun 10, 2013, at 11:46 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
> Hi, Airavata users,
>
> Do you have any advises or experiences for my previous mail about ForEach component?
> I am still struggling with generating input for ForEach component in my script :/
> Any comment will be helpful to me to use the component :)
>
> Thank you!
>
> ---------- Forwarded message ----------
> From: Heejoon Chae <he...@cs.indiana.edu>
> Date: Fri, Jun 7, 2013 at 5:44 PM
> Subject: Re: Question for Airavata For-Each component
> To: "users@airavata.apache.org" <us...@airavata.apache.org>
>
>
> Hi, Lahir,
>
> I succeeded to use ForEach component with attaching 'Input' component in front of ForEach componet with comma separated value(i.e "a,b,c") as input like in attached figure. ForEach component works well with Input component with comma separate value.
>
> Now I want to mimic what Input component does within my script as in another figure. This is required to run ForEach on generated data not on data inputed from user.
>
> I tired several ways to generate a input for ForEach component in my script including what you suggested in previous mail, but I couldn't. Please help me to solve this problem.
>
> Thank you,
> Heejoon.
>
>
> On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com> wrote:
> Following script will be useful for you..
>
> echo $@
> echo $*
> echo $#
> for arg; do
> printf '%s\n' "$arg"
> done
>
>
> Lahir
>
>
> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
> Hi Lahiru,
>
> Thank you for your answer.
>
> By the way, how to create string array in shell script, so that I can pass it to ForEach component?
>
> Thank you,
> Heejoon.
>
> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>
>> Hi Heejoon,
>>
>> We take string array as input but we invoke the script with each element of the string array multiple times, in Foreach we don't pass whole string array to the application script at once. So you really don't have to handle a string array in your script.
>>
>> Lahiru
>>
>>
>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>> Hi,
>>
>> I have quick question about Airavata For-Each component.
>>
>> It seems that ForEach component takes String array as an input and pass them to next component one by one.
>>
>> My question is how to create string array in shell script to pass it to ForEach component?
>>
>> I normally generate string result like
>>
>> echo "output=xxx"
>>
>> at the end of script and connect it as an input of next component in Airavata workflow to pass the result and let next component know the input is arrived.
>>
>> Thanks,
>> Heejoon.
>>
>>
>>
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>
>
> <Screen Shot 2013-06-07 at 3.12.05 PM.png><Screen Shot 2013-06-07 at 3.11.31 PM.png>
Fwd: Question for Airavata For-Each component
Posted by Heejoon Chae <he...@cs.indiana.edu>.
Hi, Airavata users,
Do you have any advises or experiences for my previous mail about ForEach
component?
I am still struggling with generating input for ForEach component in my
script :/
Any comment will be helpful to me to use the component :)
Thank you!
---------- Forwarded message ----------
From: Heejoon Chae <he...@cs.indiana.edu>
Date: Fri, Jun 7, 2013 at 5:44 PM
Subject: Re: Question for Airavata For-Each component
To: "users@airavata.apache.org" <us...@airavata.apache.org>
Hi, Lahir,
I succeeded to use ForEach component with attaching 'Input' component in
front of ForEach componet with comma separated value(i.e "a,b,c") as input
like in attached figure. ForEach component works well with Input component
with comma separate value.
Now I want to mimic what Input component does within my script as in
another figure. This is required to run ForEach on generated data not on
data inputed from user.
I tired several ways to generate a input for ForEach component in my script
including what you suggested in previous mail, but I couldn't. Please help
me to solve this problem.
Thank you,
Heejoon.
On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>wrote:
> Following script will be useful for you..
>
> echo $@
> echo $*
> echo $#
> for arg; do
> printf '%s\n' "$arg"
> done
>
>
> Lahir
>
>
> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>
>> Hi Lahiru,
>>
>> Thank you for your answer.
>>
>> By the way, how to create string array in shell script, so that I can
>> pass it to ForEach component?
>>
>> Thank you,
>> Heejoon.
>>
>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>>
>> Hi Heejoon,
>>
>> We take string array as input but we invoke the script with each element
>> of the string array multiple times, in Foreach we don't pass whole string
>> array to the application script at once. So you really don't have to handle
>> a string array in your script.
>>
>> Lahiru
>>
>>
>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>wrote:
>>
>>> Hi,
>>>
>>> I have quick question about Airavata For-Each component.
>>>
>>> It seems that ForEach component takes String array as an input and pass
>>> them to next component one by one.
>>>
>>> My question is how to create string array in shell script to pass it to
>>> ForEach component?
>>>
>>> I normally generate string result like
>>>
>>> echo "output=xxx"
>>>
>>> at the end of script and connect it as an input of next component in
>>> Airavata workflow to pass the result and let next component know the input
>>> is arrived.
>>>
>>> Thanks,
>>> Heejoon.
>>>
>>
>>
>>
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University
>>
>>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>
Re: Question for Airavata For-Each component
Posted by Heejoon Chae <he...@cs.indiana.edu>.
Hi, Lahir,
I succeeded to use ForEach component with attaching 'Input' component in
front of ForEach componet with comma separated value(i.e "a,b,c") as input
like in attached figure. ForEach component works well with Input component
with comma separate value.
Now I want to mimic what Input component does within my script as in
another figure. This is required to run ForEach on generated data not on
data inputed from user.
I tired several ways to generate a input for ForEach component in my script
including what you suggested in previous mail, but I couldn't. Please help
me to solve this problem.
Thank you,
Heejoon.
On Wed, Jun 5, 2013 at 11:44 PM, Lahiru Gunathilake <gl...@gmail.com>wrote:
> Following script will be useful for you..
>
> echo $@
> echo $*
> echo $#
> for arg; do
> printf '%s\n' "$arg"
> done
>
>
> Lahir
>
>
> On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
>
>> Hi Lahiru,
>>
>> Thank you for your answer.
>>
>> By the way, how to create string array in shell script, so that I can
>> pass it to ForEach component?
>>
>> Thank you,
>> Heejoon.
>>
>> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>>
>> Hi Heejoon,
>>
>> We take string array as input but we invoke the script with each element
>> of the string array multiple times, in Foreach we don't pass whole string
>> array to the application script at once. So you really don't have to handle
>> a string array in your script.
>>
>> Lahiru
>>
>>
>> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>wrote:
>>
>>> Hi,
>>>
>>> I have quick question about Airavata For-Each component.
>>>
>>> It seems that ForEach component takes String array as an input and pass
>>> them to next component one by one.
>>>
>>> My question is how to create string array in shell script to pass it to
>>> ForEach component?
>>>
>>> I normally generate string result like
>>>
>>> echo "output=xxx"
>>>
>>> at the end of script and connect it as an input of next component in
>>> Airavata workflow to pass the result and let next component know the input
>>> is arrived.
>>>
>>> Thanks,
>>> Heejoon.
>>>
>>
>>
>>
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University
>>
>>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>
Re: Question for Airavata For-Each component
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Following script will be useful for you..
echo $@
echo $*
echo $#
for arg; do
printf '%s\n' "$arg"
done
Lahir
On Wed, Jun 5, 2013 at 9:53 AM, 채희준 <ic...@gmail.com> wrote:
> Hi Lahiru,
>
> Thank you for your answer.
>
> By the way, how to create string array in shell script, so that I can pass
> it to ForEach component?
>
> Thank you,
> Heejoon.
>
> 2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
>
> Hi Heejoon,
>
> We take string array as input but we invoke the script with each element
> of the string array multiple times, in Foreach we don't pass whole string
> array to the application script at once. So you really don't have to handle
> a string array in your script.
>
> Lahiru
>
>
> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>wrote:
>
>> Hi,
>>
>> I have quick question about Airavata For-Each component.
>>
>> It seems that ForEach component takes String array as an input and pass
>> them to next component one by one.
>>
>> My question is how to create string array in shell script to pass it to
>> ForEach component?
>>
>> I normally generate string result like
>>
>> echo "output=xxx"
>>
>> at the end of script and connect it as an input of next component in
>> Airavata workflow to pass the result and let next component know the input
>> is arrived.
>>
>> Thanks,
>> Heejoon.
>>
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>
>
--
System Analyst Programmer
PTI Lab
Indiana University
Re: Question for Airavata For-Each component
Posted by 채희준 <ic...@gmail.com>.
Hi Lahiru,
Thank you for your answer.
By the way, how to create string array in shell script, so that I can pass it to ForEach component?
Thank you,
Heejoon.
2013. 6. 5. 오후 10:44 Lahiru Gunathilake <gl...@gmail.com> 작성:
> Hi Heejoon,
>
> We take string array as input but we invoke the script with each element of the string array multiple times, in Foreach we don't pass whole string array to the application script at once. So you really don't have to handle a string array in your script.
>
> Lahiru
>
>
> On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu> wrote:
>> Hi,
>>
>> I have quick question about Airavata For-Each component.
>>
>> It seems that ForEach component takes String array as an input and pass them to next component one by one.
>>
>> My question is how to create string array in shell script to pass it to ForEach component?
>>
>> I normally generate string result like
>>
>> echo "output=xxx"
>>
>> at the end of script and connect it as an input of next component in Airavata workflow to pass the result and let next component know the input is arrived.
>>
>> Thanks,
>> Heejoon.
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
Re: Question for Airavata For-Each component
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Hi Heejoon,
We take string array as input but we invoke the script with each element of
the string array multiple times, in Foreach we don't pass whole string
array to the application script at once. So you really don't have to handle
a string array in your script.
Lahiru
On Tue, Jun 4, 2013 at 10:06 PM, Heejoon Chae <he...@cs.indiana.edu>wrote:
> Hi,
>
> I have quick question about Airavata For-Each component.
>
> It seems that ForEach component takes String array as an input and pass
> them to next component one by one.
>
> My question is how to create string array in shell script to pass it to
> ForEach component?
>
> I normally generate string result like
>
> echo "output=xxx"
>
> at the end of script and connect it as an input of next component in
> Airavata workflow to pass the result and let next component know the input
> is arrived.
>
> Thanks,
> Heejoon.
>
--
System Analyst Programmer
PTI Lab
Indiana University