You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@airavata.apache.org by 안신영 <sy...@etri.re.kr> on 2013/04/03 11:00:16 UTC
How can I use File Type when I register web services?
When I select File type on application parameter, I wonder how File type is used.
I have some applications which read files and generate new file.
And second application must use the file generated by first application.
In this case, may I use File type as first application’s output type and second application’s input type?
If not, do I have to use String type to pass the name of file generated by first application?
I need a more detailed tutorials, workflow samples..
Regards,
Shinyoung
Re: How can I use File Type when I register web services?
Posted by Suresh Marru <sm...@apache.org>.
Hi Shinyoung,
I understand your confusion. Lets understand how Airavata determines outputs. Since Airavata Generic Application Factory manages applications of various types, it needs some standard way of determine outputs. The simplest approach is to have all of standard out passed to next application. But that does not work well, because the stdout can be very large, and secondly as you have seen with your applications, most of the scientific applications write to files. So for any basic primitive types, Airavata looks for name=value in stdout.
So for your bwa_aln application output, you will need to change the output type to URI. Secondly, you bwa_aln should printout the output variable name=path_to_file. So something like saiFile=/work/test_temp_created_dir/file1.output. If you cannot modify the source code of bwa_aln, the easiest solution is to wrap it with a shell script and at the end of the shell script you can echo these values.
An alternative mechanism is to write any files you need to the outputData sub directory. As you can see the working directory created it will have two sub directories, inputData and outputData. If name=value is not found in stdout, then Airavata will pick any files within the outputData and assume they are the required files.
If you use String as the output type, then the value /work/test_temp_created_dir/file1.output will be passed to second application. If URL is selected then it will be passed as a file. The later one is very useful if your second application is running on a different machine. In which case Airavata will first move the file and then execute the second application.
Hope this helps,
Suresh
On Apr 3, 2013, at 10:36 PM, 안신영 <sy...@etri.re.kr> wrote:
> Thanks Suresh!
>
> But, I’m not clear about using arguments.
>
> As I understand, File Type should be used for file staging from the node of airavata server to node of globus server.
> Am I correct?
>
> If so, I want to talk about my use case :
> First application(bwa_aln) generate file, second application(bwa_sampe) should start after execution of first application and should receive the file name, which will be generated by first application.
> The file name which first application generate is input argument of first application.
> As I understand, for the sequential execution of these two applications, two application should be linked with each other so that second application get the output argument of first application.
> Next figure shows my workflow. I want to use String type as first application’s output type and use String type as second application’s input type.
> The String type argument means filename(saiFile below).
>
> <image001.png>
> such workflow returns errors as follows
>
> [ERROR] Data for the output parameter 'saiFile' was not found
> org.apache.airavata.core.gfac.exception.ProviderException: Data for the output parameter 'saiFile' was not found
> at org.apache.airavata.core.gfac.provider.impl.GramProvider.processOutput(GramProvider.java:369)
> at org.apache.airavata.core.gfac.provider.AbstractProvider.execute(AbstractProvider.java:76)
> at org.apache.airavata.core.gfac.services.impl.AbstractSimpleService.execute(AbstractSimpleService.java:118)
> at org.apache.airavata.core.gfac.GfacAPI.gridJobSubmit(GfacAPI.java:140)
> at org.apache.airavata.xbaya.invoker.EmbeddedGFacInvoker.invoke(EmbeddedGFacInvoker.java:256)
> at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.handleWSComponent(WorkflowInterpreter.java:749)
> at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.executeDynamically(WorkflowInterpreter.java:533)
> at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.scheduleDynamically(WorkflowInterpreter.java:218)
> at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.executeWorkflow(WorkflowInterpretorSkeleton.java:389)
> at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.access$400(WorkflowInterpretorSkeleton.java:87)
> at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton$2.run(WorkflowInterpretorSkeleton.java:382)
> at java.lang.Thread.run(Thread.java:636)
>
> when the output type of first application is String, Why Airavata throw errors?
> How do I have to handle these error?
>
> Thanks in advance.
> Shinyoung.
> From: Suresh Marru [mailto:smarru@apache.org]
> Sent: Wednesday, April 03, 2013 9:44 PM
> To: users@airavata.apache.org
> Subject: Re: How can I use File Type when I register web services?
>
> Hi Shinyoug,
>
> Please read this thread if it will answer your questions - http://markmail.org/thread/hd7azhp7w7o7eqyq
>
> Meanwhile I will work on a wiki page to address specifically your questions. These are good FAQ's. We will appreciate if you can contribute from your learning as well.
>
> Cheers,
> Suresh
>
> On Apr 3, 2013, at 5:00 AM, 안신영 <sy...@etri.re.kr> wrote:
>
>
> When I select File type on application parameter, I wonder how File type is used.
>
> I have some applications which read files and generate new file.
> And second application must use the file generated by first application.
> In this case, may I use File type as first application’s output type and second application’s input type?
> If not, do I have to use String type to pass the name of file generated by first application?
>
> I need a more detailed tutorials, workflow samples..
>
> Regards,
> Shinyoung
>
RE: How can I use File Type when I register web services?
Posted by 안신영 <sy...@etri.re.kr>.
Thanks Suresh!
But, I’m not clear about using arguments.
As I understand, File Type should be used for file staging from the node of airavata server to node of globus server.
Am I correct?
If so, I want to talk about my use case :
First application(bwa_aln) generate file, second application(bwa_sampe) should start after execution of first application and should receive the file name, which will be generated by first application.
The file name which first application generate is input argument of first application.
As I understand, for the sequential execution of these two applications, two application should be linked with each other so that second application get the output argument of first application.
Next figure shows my workflow. I want to use String type as first application’s output type and use String type as second application’s input type.
The String type argument means filename(saiFile below).
[cid:image001.png@01CE3127.6BDDB2E0]
such workflow returns errors as follows
[ERROR] Data for the output parameter 'saiFile' was not found
org.apache.airavata.core.gfac.exception.ProviderException: Data for the output parameter 'saiFile' was not found
at org.apache.airavata.core.gfac.provider.impl.GramProvider.processOutput(GramProvider.java:369)
at org.apache.airavata.core.gfac.provider.AbstractProvider.execute(AbstractProvider.java:76)
at org.apache.airavata.core.gfac.services.impl.AbstractSimpleService.execute(AbstractSimpleService.java:118)
at org.apache.airavata.core.gfac.GfacAPI.gridJobSubmit(GfacAPI.java:140)
at org.apache.airavata.xbaya.invoker.EmbeddedGFacInvoker.invoke(EmbeddedGFacInvoker.java:256)
at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.handleWSComponent(WorkflowInterpreter.java:749)
at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.executeDynamically(WorkflowInterpreter.java:533)
at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.scheduleDynamically(WorkflowInterpreter.java:218)
at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.executeWorkflow(WorkflowInterpretorSkeleton.java:389)
at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.access$400(WorkflowInterpretorSkeleton.java:87)
at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton$2.run(WorkflowInterpretorSkeleton.java:382)
at java.lang.Thread.run(Thread.java:636)
when the output type of first application is String, Why Airavata throw errors?
How do I have to handle these error?
Thanks in advance.
Shinyoung.
From: Suresh Marru [mailto:smarru@apache.org]
Sent: Wednesday, April 03, 2013 9:44 PM
To: users@airavata.apache.org
Subject: Re: How can I use File Type when I register web services?
Hi Shinyoug,
Please read this thread if it will answer your questions - http://markmail.org/thread/hd7azhp7w7o7eqyq
Meanwhile I will work on a wiki page to address specifically your questions. These are good FAQ's. We will appreciate if you can contribute from your learning as well.
Cheers,
Suresh
On Apr 3, 2013, at 5:00 AM, 안신영 <sy...@etri.re.kr>> wrote:
When I select File type on application parameter, I wonder how File type is used.
I have some applications which read files and generate new file.
And second application must use the file generated by first application.
In this case, may I use File type as first application’s output type and second application’s input type?
If not, do I have to use String type to pass the name of file generated by first application?
I need a more detailed tutorials, workflow samples..
Regards,
Shinyoung
Re: How can I use File Type when I register web services?
Posted by Suresh Marru <sm...@apache.org>.
Hi Shinyoug,
Please read this thread if it will answer your questions - http://markmail.org/thread/hd7azhp7w7o7eqyq
Meanwhile I will work on a wiki page to address specifically your questions. These are good FAQ's. We will appreciate if you can contribute from your learning as well.
Cheers,
Suresh
On Apr 3, 2013, at 5:00 AM, 안신영 <sy...@etri.re.kr> wrote:
> When I select File type on application parameter, I wonder how File type is used.
>
> I have some applications which read files and generate new file.
> And second application must use the file generated by first application.
> In this case, may I use File type as first application’s output type and second application’s input type?
> If not, do I have to use String type to pass the name of file generated by first application?
>
> I need a more detailed tutorials, workflow samples..
>
> Regards,
> Shinyoung
>