You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@taverna.apache.org by Daniele Tartarini <d....@sheffield.ac.uk> on 2016/04/26 23:39:32 UTC
Taverna Server REST api provide input files
Hi,
I am using a Taverna server to run a workflow based on Taverna tools
service.
I can run execution correctly following the API here:
http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface
The test workflow is very simple, taking a single input file.
I noted that when I upload to the server the input file using:
POST to
https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be//wd/IN
and then associating the port to the file using:
PUT to
https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/input/input/input_file_text
with data
<t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
<t2sr:file>IN/myfile.txt</t2sr:file> </t2sr:runInput>
execution is correct but input files are copied from the
/tmp/42d06756-b9c4-45ef-b69d-a141c69499be
to the Taverna Tool service working folder 'usecase....dir'
I would like to avoid this copy because I would be dealing to substantial
inputs and on a parameter sweep can affect performance.
In Taverna Workbench this issue is solved in a elegant way using the "Set
File Location" tab in the form input window. Files are not copied but
replaced by symbolic links.
*How can I achieve the same result via Taverna server API or a baclava (I
noted the Set file location information is not saved inside the baclava)?*
I tried to use the Taverna server API setting the input as reference as
documented in:
http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface section:
"Using a File Already on the Taverna Server Installation"
Using:
PUT
https://myserver.ac.uk:8444/taverna-server/rest/runs/88f79b84-6451-4aed-b151-6b051328d922
/input/input/input_file_text
with data ({'content-type': 'application/xml'}):
<t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
<t2sr:reference>
https://myserver.ac.uk:8444/taverna-server/rest/runs/88f79b84-6451-4aed-b151-6b051328d922/wd/IN/myfile.txt
</t2sr:reference>
</t2sr:runInput>
I get the following 403 error: filename may not be absolute
*Any Idea how to solve both issues?*
Many thanks in advance.
Cheers
Daniele
--
--
Daniele Tartarini
Post-Doctoral Research Associate
Dept. Mechanical Engineering &
INSIGNEO, institute for *in silico* medicine,
University of Sheffield, Sheffield, UK
linkedIn <http://uk.linkedin.com/in/danieletartarini>
Re: Fwd: Taverna Server REST api provide input files
Posted by Stian Soiland-Reyes <st...@apache.org>.
This was also discussed on dev@taverna - we found it was a bug in the
executeworkflow command line.
> Tracked as https <https://issues.apache.org/jira/browse/TAVERNA-964>://
<https://issues.apache.org/jira/browse/TAVERNA-964>issues.apache.org
<https://issues.apache.org/jira/browse/TAVERNA-964>/
<https://issues.apache.org/jira/browse/TAVERNA-964>jira
<https://issues.apache.org/jira/browse/TAVERNA-964>/browse/TAVERNA-964
<https://issues.apache.org/jira/browse/TAVERNA-964>
It should not be too complicated to fix - perhaps you want to have a go on
a pull request? :-)
On 29 Apr 2016 11:26 a.m., "Daniele Tartarini" <d....@sheffield.ac.uk>
wrote:
> Hi,
>
> I am using a Taverna server 2.5.4 to run a workflow based on Taverna tools
> service.
>
> I can run execution correctly following the REST API here:
> http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface
>
> The workflow I'm using is very simple and takes a single input file.
>
> I noted that when I upload to Taverna server input file using:
> POST to
> https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/wd/IN
>
> and then associating the port to the file using:
>
> PUT to
> https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/input/input/input_file_text
> with data
> <t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
> <t2sr:file>IN/myfile.txt</t2sr:file> </t2sr:runInput>
>
> execution is correct but input files are hard copied from the
> /tmp/42d06756-b9c4-45ef-b69d-a141c69499be
> to the Taverna Tool service working folder 'usecase....dir'
>
> I would like to avoid this copy because I would be dealing to substantial
> inputs and on a parameter sweep can affect performance.
>
> In Taverna Workbench this issue is solved in a elegant way using the "Set
> File Location" tab in the form input window. Files are not copied but
> replaced by symbolic links.
>
> *How can I achieve the same result via Taverna server API or a baclava
> file (I noted the Set file location information is not saved inside the
> baclava)?*
>
> I tried to use the Taverna server API setting the input as reference as
> documented in:
> http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface
> section: "Using a File Already on the Taverna Server Installation"
>
> Using:
> PUT
> https://myserver.ac.uk:8444/taverna-server/rest/runs/88f79b84-6451-4aed-b151-6b051328d922
> /input/input/input_file_text
>
> with data ({'content-type': 'application/xml'}):
>
> <t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/
> ">
> <t2sr:reference>
>
> https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/
> wd/IN/myfile.txt
> </t2sr:reference>
> </t2sr:runInput>
>
>
> I get the following 403 error: filename may not be absolute
>
> *Any Idea how to solve both issues?*
>
> Many thanks in advance.
>
> Cheers
>
> Daniele
>
>
> --
> --
> Daniele Tartarini
>
> Post-Doctoral Research Associate
> Dept. Mechanical Engineering &
> INSIGNEO, institute for *in silico* medicine,
> University of Sheffield, Sheffield, UK
> linkedIn <http://uk.linkedin.com/in/danieletartarini>
>
>
>
>
> --
> --
> Daniele Tartarini
>
> Post-Doctoral Research Associate
> Dept. Mechanical Engineering &
> INSIGNEO, institute for *in silico* medicine,
> University of Sheffield, Sheffield, UK
> linkedIn <http://uk.linkedin.com/in/danieletartarini>
>
>
Fwd: Taverna Server REST api provide input files
Posted by Daniele Tartarini <d....@sheffield.ac.uk>.
Hi,
I am using a Taverna server 2.5.4 to run a workflow based on Taverna tools
service.
I can run execution correctly following the REST API here:
http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface
The workflow I'm using is very simple and takes a single input file.
I noted that when I upload to Taverna server input file using:
POST to
https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/wd/IN
and then associating the port to the file using:
PUT to
https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/input/input/input_file_text
with data
<t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
<t2sr:file>IN/myfile.txt</t2sr:file> </t2sr:runInput>
execution is correct but input files are hard copied from the
/tmp/42d06756-b9c4-45ef-b69d-a141c69499be
to the Taverna Tool service working folder 'usecase....dir'
I would like to avoid this copy because I would be dealing to substantial
inputs and on a parameter sweep can affect performance.
In Taverna Workbench this issue is solved in a elegant way using the "Set
File Location" tab in the form input window. Files are not copied but
replaced by symbolic links.
*How can I achieve the same result via Taverna server API or a baclava file
(I noted the Set file location information is not saved inside the
baclava)?*
I tried to use the Taverna server API setting the input as reference as
documented in:
http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface section:
"Using a File Already on the Taverna Server Installation"
Using:
PUT
https://myserver.ac.uk:8444/taverna-server/rest/runs/88f79b84-6451-4aed-b151-6b051328d922
/input/input/input_file_text
with data ({'content-type': 'application/xml'}):
<t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
<t2sr:reference>
https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/
wd/IN/myfile.txt
</t2sr:reference>
</t2sr:runInput>
I get the following 403 error: filename may not be absolute
*Any Idea how to solve both issues?*
Many thanks in advance.
Cheers
Daniele
--
--
Daniele Tartarini
Post-Doctoral Research Associate
Dept. Mechanical Engineering &
INSIGNEO, institute for *in silico* medicine,
University of Sheffield, Sheffield, UK
linkedIn <http://uk.linkedin.com/in/danieletartarini>
--
--
Daniele Tartarini
Post-Doctoral Research Associate
Dept. Mechanical Engineering &
INSIGNEO, institute for *in silico* medicine,
University of Sheffield, Sheffield, UK
linkedIn <http://uk.linkedin.com/in/danieletartarini>
Re: Taverna Server REST api provide input files
Posted by Stian Soiland-Reyes <st...@apache.org>.
Tracked as https://issues.apache.org/jira/browse/TAVERNA-964
Any takers? :)
On 27 April 2016 at 11:08, Stian Soiland-Reyes <st...@apache.org> wrote:
> Thanks for reporting this.
>
> I think the issue is in the executeworkflow command line tool, as I
> can replicate this with also with executeworkflow -inputfile - e.g. a
> Tool activity with "ls -al `pwd`/fred"
> will output in the Workbench:
>
> lrwxrwxrwx 1 stain stain 17 Apr 27 10:52
> /tmp/usecase2549370560835453456dir/fred -> /home/stain/2.ttl
>
> but run as executeworkflow fred.t2flow -inputfile fred /tmp/oa.html
>
> it will save a copy:
>
> -rw-rw-r-- 1 stain stain 171344 Apr 27 10:53
> /tmp/usecase5592972701647292140dir/fred
>
>
> so it seems the command line is not providing the input files as File
> references for the Tool activity to recognize - thus Tool is forced to
> save them as new files.
>
> I think in Taverna 3 the file will actually be copied twice - once
> into the input DataBundle, and then out to the tool activity.
>
>
>
> Relevant code:
> https://github.com/apache/incubator-taverna-common-activities/blob/master/taverna-external-tool-activity/src/main/java/de/uni_luebeck/inb/knowarc/usecases/invocation/local/LocalUseCaseInvocation.java#L142
>
> https://github.com/apache/incubator-taverna-commandline/blob/master/taverna-commandline-common/src/main/java/org/apache/taverna/commandline/CommandLineTool.java#L234
>
> https://github.com/apache/incubator-taverna-commandline/blob/master/taverna-commandline-common/src/main/java/org/apache/taverna/commandline/data/InputsHandler.java#L168
>
> 2.5 code as well:
>
> https://github.com/apache/incubator-taverna-commandline/blob/old/taverna-commandline-1.5/taverna-commandline-common/src/main/java/net/sf/taverna/t2/commandline/data/InputsHandler.java#L190
>
> Both of these seems to read the file into memory (!) - even though (in
> Taverna 3) setValue() supports File and URL forwarded on to
> DataBundles.setReference().
>
> I think URL can then be converted back to File by Tool if it starts
> with file:/// - worth checking in ToolActivity.
>
>
> On 26 April 2016 at 22:39, Daniele Tartarini
> <d....@sheffield.ac.uk> wrote:
>> Hi,
>>
>> I am using a Taverna server to run a workflow based on Taverna tools
>> service.
>>
>> I can run execution correctly following the API here:
>> http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface
>>
>> The test workflow is very simple, taking a single input file.
>>
>> I noted that when I upload to the server the input file using:
>> POST to
>> https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be//wd/IN
>>
>> and then associating the port to the file using:
>>
>> PUT to
>> https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/input/input/input_file_text
>> with data
>> <t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
>> <t2sr:file>IN/myfile.txt</t2sr:file> </t2sr:runInput>
>>
>> execution is correct but input files are copied from the
>> /tmp/42d06756-b9c4-45ef-b69d-a141c69499be
>> to the Taverna Tool service working folder 'usecase....dir'
>>
>> I would like to avoid this copy because I would be dealing to substantial
>> inputs and on a parameter sweep can affect performance.
>>
>> In Taverna Workbench this issue is solved in a elegant way using the "Set
>> File Location" tab in the form input window. Files are not copied but
>> replaced by symbolic links.
>>
>> *How can I achieve the same result via Taverna server API or a baclava (I
>> noted the Set file location information is not saved inside the baclava)?*
>>
>> I tried to use the Taverna server API setting the input as reference as
>> documented in:
>> http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface section:
>> "Using a File Already on the Taverna Server Installation"
>>
>> Using:
>> PUT
>> https://myserver.ac.uk:8444/taverna-server/rest/runs/88f79b84-6451-4aed-b151-6b051328d922
>> /input/input/input_file_text
>>
>> with data ({'content-type': 'application/xml'}):
>>
>> <t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
>> <t2sr:reference>
>>
>> https://myserver.ac.uk:8444/taverna-server/rest/runs/88f79b84-6451-4aed-b151-6b051328d922/wd/IN/myfile.txt
>> </t2sr:reference>
>> </t2sr:runInput>
>>
>>
>> I get the following 403 error: filename may not be absolute
>>
>> *Any Idea how to solve both issues?*
>>
>> Many thanks in advance.
>>
>> Cheers
>>
>> Daniele
>>
>>
>> --
>> --
>> Daniele Tartarini
>>
>> Post-Doctoral Research Associate
>> Dept. Mechanical Engineering &
>> INSIGNEO, institute for *in silico* medicine,
>> University of Sheffield, Sheffield, UK
>> linkedIn <http://uk.linkedin.com/in/danieletartarini>
>
>
>
> --
> Stian Soiland-Reyes
> Apache Taverna (incubating), Apache Commons RDF (incubating)
> http://orcid.org/0000-0001-9842-9718
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
Re: Taverna Server REST api provide input files
Posted by Stian Soiland-Reyes <st...@apache.org>.
Thanks for reporting this.
I think the issue is in the executeworkflow command line tool, as I
can replicate this with also with executeworkflow -inputfile - e.g. a
Tool activity with "ls -al `pwd`/fred"
will output in the Workbench:
lrwxrwxrwx 1 stain stain 17 Apr 27 10:52
/tmp/usecase2549370560835453456dir/fred -> /home/stain/2.ttl
but run as executeworkflow fred.t2flow -inputfile fred /tmp/oa.html
it will save a copy:
-rw-rw-r-- 1 stain stain 171344 Apr 27 10:53
/tmp/usecase5592972701647292140dir/fred
so it seems the command line is not providing the input files as File
references for the Tool activity to recognize - thus Tool is forced to
save them as new files.
I think in Taverna 3 the file will actually be copied twice - once
into the input DataBundle, and then out to the tool activity.
Relevant code:
https://github.com/apache/incubator-taverna-common-activities/blob/master/taverna-external-tool-activity/src/main/java/de/uni_luebeck/inb/knowarc/usecases/invocation/local/LocalUseCaseInvocation.java#L142
https://github.com/apache/incubator-taverna-commandline/blob/master/taverna-commandline-common/src/main/java/org/apache/taverna/commandline/CommandLineTool.java#L234
https://github.com/apache/incubator-taverna-commandline/blob/master/taverna-commandline-common/src/main/java/org/apache/taverna/commandline/data/InputsHandler.java#L168
2.5 code as well:
https://github.com/apache/incubator-taverna-commandline/blob/old/taverna-commandline-1.5/taverna-commandline-common/src/main/java/net/sf/taverna/t2/commandline/data/InputsHandler.java#L190
Both of these seems to read the file into memory (!) - even though (in
Taverna 3) setValue() supports File and URL forwarded on to
DataBundles.setReference().
I think URL can then be converted back to File by Tool if it starts
with file:/// - worth checking in ToolActivity.
On 26 April 2016 at 22:39, Daniele Tartarini
<d....@sheffield.ac.uk> wrote:
> Hi,
>
> I am using a Taverna server to run a workflow based on Taverna tools
> service.
>
> I can run execution correctly following the API here:
> http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface
>
> The test workflow is very simple, taking a single input file.
>
> I noted that when I upload to the server the input file using:
> POST to
> https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be//wd/IN
>
> and then associating the port to the file using:
>
> PUT to
> https://myserver.ac.uk/taverna-server/rest/runs/42d06756-b9c4-45ef-b69d-a141c69499be/input/input/input_file_text
> with data
> <t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
> <t2sr:file>IN/myfile.txt</t2sr:file> </t2sr:runInput>
>
> execution is correct but input files are copied from the
> /tmp/42d06756-b9c4-45ef-b69d-a141c69499be
> to the Taverna Tool service working folder 'usecase....dir'
>
> I would like to avoid this copy because I would be dealing to substantial
> inputs and on a parameter sweep can affect performance.
>
> In Taverna Workbench this issue is solved in a elegant way using the "Set
> File Location" tab in the form input window. Files are not copied but
> replaced by symbolic links.
>
> *How can I achieve the same result via Taverna server API or a baclava (I
> noted the Set file location information is not saved inside the baclava)?*
>
> I tried to use the Taverna server API setting the input as reference as
> documented in:
> http://dev.mygrid.org.uk/wiki/display/tav250/Conceptual+Interface section:
> "Using a File Already on the Taverna Server Installation"
>
> Using:
> PUT
> https://myserver.ac.uk:8444/taverna-server/rest/runs/88f79b84-6451-4aed-b151-6b051328d922
> /input/input/input_file_text
>
> with data ({'content-type': 'application/xml'}):
>
> <t2sr:runInput xmlns:t2sr="http://ns.taverna.org.uk/2010/xml/server/rest/">
> <t2sr:reference>
>
> https://myserver.ac.uk:8444/taverna-server/rest/runs/88f79b84-6451-4aed-b151-6b051328d922/wd/IN/myfile.txt
> </t2sr:reference>
> </t2sr:runInput>
>
>
> I get the following 403 error: filename may not be absolute
>
> *Any Idea how to solve both issues?*
>
> Many thanks in advance.
>
> Cheers
>
> Daniele
>
>
> --
> --
> Daniele Tartarini
>
> Post-Doctoral Research Associate
> Dept. Mechanical Engineering &
> INSIGNEO, institute for *in silico* medicine,
> University of Sheffield, Sheffield, UK
> linkedIn <http://uk.linkedin.com/in/danieletartarini>
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718