You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@taverna.apache.org by Menaka Madushanka <me...@gmail.com> on 2015/03/07 16:48:27 UTC
GSOC 2015 - Taverna: Language command line tool
Hi,
I downloaded the taverna language from GIT repository and built it.
And I went through the references that were given in
https://issues.apache.org/jira/browse/COMDEV-120?filter=12330297
So, to clarify more about the project I'd like to get some ideas about what
are the command line options that taverna has.
Also I need to know whether there should be a separate command line
interface.
Thank you very much.
Cheers...!!!
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Menaka Madushanka <me...@gmail.com>.
Thank you very much Christian....!!!
[?]
On 9 March 2015 at 14:30, Christian Brenninkmeijer <
christian.brenninkmeijer@manchester.ac.uk> wrote:
> Have a look at:
> http://taverna.incubator.apache.org/documentation/command-line/
> This list some of the options.
>
> Or run:
> sh executeworkflow.sh -help
>
> to get the full list
>
> Christian
> ________________________________________
> From: Menaka Madushanka [menaka12350@gmail.com]
> Sent: Saturday, March 07, 2015 3:48 PM
> To: dev@taverna.incubator.apache.org
> Cc: stain@apache.org
> Subject: GSOC 2015 - Taverna: Language command line tool
>
> Hi,
> I downloaded the taverna language from GIT repository and built it.
> And I went through the references that were given in
> https://issues.apache.org/jira/browse/COMDEV-120?filter=12330297
>
> So, to clarify more about the project I'd like to get some ideas about what
> are the command line options that taverna has.
>
> Also I need to know whether there should be a separate command line
> interface.
>
> Thank you very much.
>
> Cheers...!!!
>
RE: GSOC 2015 - Taverna: Language command line tool
Posted by Christian Brenninkmeijer <ch...@manchester.ac.uk>.
Have a look at:
http://taverna.incubator.apache.org/documentation/command-line/
This list some of the options.
Or run:
sh executeworkflow.sh -help
to get the full list
Christian
________________________________________
From: Menaka Madushanka [menaka12350@gmail.com]
Sent: Saturday, March 07, 2015 3:48 PM
To: dev@taverna.incubator.apache.org
Cc: stain@apache.org
Subject: GSOC 2015 - Taverna: Language command line tool
Hi,
I downloaded the taverna language from GIT repository and built it.
And I went through the references that were given in
https://issues.apache.org/jira/browse/COMDEV-120?filter=12330297
So, to clarify more about the project I'd like to get some ideas about what
are the command line options that taverna has.
Also I need to know whether there should be a separate command line
interface.
Thank you very much.
Cheers...!!!
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Menaka Madushanka <me...@gmail.com>.
Hi Stain,
I think if we have an idea about which commands have more priority and what
are the commands that are used in more frequently,
we can easily simplify the problem.
Cheers.
Menaka
On 11 March 2015 at 16:44, Stian Soiland-Reyes <st...@apache.org> wrote:
> ... Let's try to keep the technical bit of this thread on dev@taverna
> - but feel free to contact me personally for private matters such as
> your commitments.
>
>
> On 10 March 2015 at 19:25, Menaka Madushanka <me...@gmail.com>
> wrote:
>
> > If I understood correctly,
> > I have to implement the generalized version of the command line tools by
> the
> > mid evaluation
> >
> https://github.com/apache/incubator-taverna-language/tree/master/taverna-scufl2-examples
> >
> https://github.com/apache/incubator-taverna-language/tree/master/taverna-scufl2-wfdesc
> > https://github.com/stain/ro-combine-archive
>
> Right - that sounds like a good plan. Once we have a prototype up and
> running we can see better what fits well.
>
> This should include writing some documentation on command line
> options, example usage etc.
>
>
> > After that update ExecuteWorkflow....
>
> Perhaps we should flesh out a more detailed plan for ExecuteWorkflow.
>
> Having some kind of -verbose mode with proper logging was mentioned -
> I can help you show how to hook into the platform to get
> notifications. Perhaps this needs some kind of verbosity level as
> some workflows can be very active and thus be very noisy if everything
> is logged.
>
>
> Have you got any other suggestions? For instance on simplifying the
> command line options and perhaps moving some of these to a config
> file? Making it a bit more unix-like perhaps.
>
>
>
> stain@biggie-utopic
> :~/src/taverna/incubator-taverna-commandline/taverna-commandline-product/target/apache-taverna-commandline-3.1.0.incubating-SNAPSHOT-dev/apache-taverna-commandline-3.1.0.incubating-SNAPSHOT$
> ./executeworkflow.sh --help
> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=400m;
> support was removed in 8.0
> usage: executeworkflow [options] [workflow]
> -bundle <bundle> Save outputs to a new Workflow
> Run Bundle (zip).
> -clientserver Connect as a client to a derby
> server instance.
> -cmdir <directory path> Absolute path to a directory
> where Credential Manager's files
> (keystore and truststore) are
> located.
> -cmpassword Indicate that the master password
> for Credential Manager will be
> provided on standard input.
> -dbproperties <filename> Load a properties file to
> configure the database.
> -embedded Connect to an embedded Derby
> database. This can prevent
> mulitple invocations.
> -help Display comprehensive help
> information.
> -inmemory Run the workflow with data stored
> in-memory rather than in a
> database (this is the default
> option). This can give
> performance inprovements, at the
> cost of overall memory usage.
> -inputdelimiter <inputname delimiter> Cause an inputvalue or inputfile
> to be split into a list according
> to the delimiter. The associated
> workflow input must be expected
> to receive a list.
> -inputdoc <document> Load inputs from a Baclava
> document.
> -inputfile <inputname filename> Load the named input from file or
> URL.
> -inputvalue <inputname value> Directly use the value for the
> named input.
> -logfile <filename> The logfile to which more verbose
> logging will be written to.
> -outputdir <directory> Save outputs as files in
> directory, default is to make a
> new directory
> workflowName_output.
> -port <portnumber> The port that the database is
> running on. If set requested to
> start its own internal server,
> this is the start port that will
> be used.
> -provenance Generate provenance information
> and store it in the database.
> -startdb Automatically start an internal
> Derby database server.
> By default, the workflow is executed using the -inmemory option, and the
> results are written out to a directory named after the workflow name.
>
> If this directory already exists then a new directory is created, and
> appended with _<n>, where n is incremented to the next available index.
>
> Results are written out to files named after the output port for that
> result.
> If a result is composed of lists, then a directory is created for the
> output
> port and individual list items are named after the list element index
> (with 1
> being the first index). The the output is the result of an error, the
> filename
> is appended with '.error'.
>
> You can provide your own output directory with the -outputdir option. There
> will be an error if the directory already exists.
>
> You can also record your results to a Baclava document using -outputdoc
> option. The document will be overwritten if it already exists.
>
> Inputs can be provided in three ways. Both -inputfile and -inputvalue
> options
> can be used together; -inputdoc option must be used on its own. -inputfile
> and
> -inputvalue options both take two additional arguments, the name of the
> port
> for the input, and either a file containing the input data, or the input
> value
> itself respectively.
>
> If one of more of your workflow inputs is a list, you can create a list
> input by using the -inputdelimiter option, which may be used with either
> -inputfile or -inputvalue. This option takes two parameters - an input name
> and the delimiter by which to split the input into a list.
>
> The delimiter may be a simple character, such as a comma or a new-line
> character, or a regular expression. The input string, or file, will then be
> converted into a list being split by the delimiter specified. Make sure to
> put the delimiter character in quotes as it may be interpreted by the shell
> as a special character, e.g. ;.
>
> If a list of greater depth (i.e. a list or lists or deeper) is required
> then
> you will need to use the -inputdoc option. However, if you provide an
> input
> of lower depth to that required, then it will automatically be wrapped in
> one
> or more lists up to the required depth. Providing an input of greater depth
> than that required will result in an error.
>
> If a workflow has a high memory requirement, then it may be better to run
> it
> using a database to store data rather than storing it in memory, which is
> the
> default option. There are three options for using a database:
>
> -embedded option, runs with an embedded database. This is slightly faster
> than
> the -clientserver option (below), but has the limitation that only one
> executeworkflow script may be executed simultaneously.
>
> -clientserver option allows the workflow to be executed backed by the
> database
> running as a server. By default a database is not started for you, but may
> be
> started using -startdb option.
>
> -startdb option starts a database. It may be used without providing a
> workflow
> to allow a database to be started separately, allowing multiple
> simultaneous
> executeworkflow script runs.
>
> More advanced database configurations can be specified using -dbproperties
> option, allowing you to take full control over the database used. This
> takes a
> second argument, the filename of the properties file, for which the
> following
> example contains the default settings:
>
> in_memory = true
> provenance = false
> connector = derby
> port = 1527
> dialect = org.hibernate.dialect.DerbyDialect
> start_derby = false
> driver = org.apache.derby.jdbc.EmbeddedDriver
> jdbcuri = jdbc:derby:t2-database;create=true;upgrade=true
>
> Note that when using -dbproperties together with other options, the other
> options take precedence.
>
> -cmdir option lets you specify an absolute path to a directory where
> Credential Manager's files (keystore and truststore - containing user's
> credentials and trusted certificates for accessing secure services) are
> stored.
> If not specified and the workflow requires access to these files, Taverna
> will
> try to find them in the default location in <TAVERNA_HOME>/security
> somewhere
> inside user's home directory (depending on the platform).
>
> -cmpassword option can be used to tell Taverna to expect the password for
> the
> Credential Manager on standard input. If the password is not piped in,
> Taverna
> will prompt you for it in the terminal and block until it is entered. Do
> not
> enter your password in the command line! If -cmpassword option is not
> specified
> and -cmdir option is used, Taverna will try to find the password in a
> special
> file password.txt in the directory specified with -cmdir option.
>
>
>
>
>
>
> --
> Stian Soiland-Reyes
> Apache Taverna (incubating), Apache Commons RDF (incubating)
> http://orcid.org/0000-0001-9842-9718
>
--
Menaka Madushanka Jayawardena
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Menaka Madushanka <me...@gmail.com>.
Hi,
I'd like to have some guidelines on, how the project proposal should be
structured.
Thank you very much.
Menaka
On 12 March 2015 at 19:36, Menaka Madushanka <me...@gmail.com> wrote:
> Thank you very much Andy,
>
> Actually the mistake was done by me. :(
> I'll correct that and will not do again.
>
> It is a grate pleasure for me to talk to you.
>
> I also like to learn new cultures and respect them. :)
>
> Thank you very much.
> Menaka
>
>
>
> On 12 March 2015 at 18:46, Andy Seaborne <an...@apache.org> wrote:
>
>> On 11/03/15 20:18, Menaka Madushanka wrote:
>>
>>> Thank you very much Andy for showing me the correct way.
>>>
>>> What happened was, I was contacting my mentor Mr: Stain to discuss about
>>>
>>
>> It's "Stian" - it's his given name, the name is Norwegian I think. And
>> sometimes he uses "Stain" as a userid. Confuses everyone!
>>
>> this matter. (I think this is a personal matter for me - my academic
>>> works, courses, etc)
>>>
>>
>> If you prefer that on private, then that's fine.
>>
>> But, for his email, I had sent the reply to dev@ mailing list.
>>> That was a mistake done by me.
>>>
>>
>> If you prefer, on private ask him not to send it on to dev@ It's your
>> email, not his!
>>
>> At Apache, I do most of my contribution to Jena, the RDF framework.
>> I'd send my holiday plans to Jena'sp private list not a public one.
>>
>> And any technical/general matter about my project, I'll discuss on dev
>>> mailing list.
>>>
>>> I'm really sorry for any inconvenience.
>>>
>>
>> No inconvenience at all. As one of the projects incubator mentors, it's
>> my responsibility to help out.
>>
>> If there is anything else you'd like to chat about do email me privately.
>>
>> I've had Indian and Chinese students before when I was an academic
>> researcher. What I leant is that there are cultural differences, and
>> differences within those as well. I'm always learning about new cultures.
>>
>> (I'm in the UK as you can tell by the timestamps on my emails - and
>> spelling of some words like "colour" and "maths".)
>>
>> Good luck and best wishes for GSoC,
>> Andy
>>
>>
>>> Thank you,
>>> Menaka
>>>
>>>
>>> On 12 March 2015 at 01:36, Andy Seaborne <andy@apache.org
>>> <ma...@apache.org>> wrote:
>>>
>>> On 11/03/15 19:40, Menaka Madushanka wrote:
>>>
>>> I'm sorry.
>>> I could not understand what you meant by
>>>
>>>
>>> Email should go on the community dev@ list unless there is a reason
>>> why not.
>>>
>>> It's part of the Apache culture and how remote, geographically
>>> separated people form a team.
>>>
>>> Andy
>>>
>>>
>>> Apache rule 1:
>>> On a mailing list!
>>>
>>> On 12 March 2015 at 00:59, Andy Seaborne <andy@apache.org
>>> <ma...@apache.org>
>>> <mailto:andy@apache.org <ma...@apache.org>>> wrote:
>>>
>>> Apache rule 1:
>>>
>>> On a mailing list!
>>>
>>> Andy
>>>
>>> I'm a mentor for the Taverna project.
>>>
>>> On 11/03/15 19:23, Menaka Madushanka wrote:
>>>
>>> Hi Andy,
>>>
>>> Thank you very much for your clarification.
>>> I'm talking to my mentor about this. It is seems like
>>> I'd have no
>>> disturbances with academic activities during summer
>>> period.
>>> How ever I know now that, I have to work hard and
>>> manage both. :-)
>>>
>>> Thank you very much.
>>> Cheers.
>>>
>>> On 12 March 2015 at 00:31, Andy Seaborne
>>> <andy@apache.org <ma...@apache.org>
>>> <mailto:andy@apache.org <ma...@apache.org>>
>>> <mailto:andy@apache.org <ma...@apache.org>
>>> <mailto:andy@apache.org <ma...@apache.org>>>> wrote:
>>>
>>> Having exams that overlap with the early part of
>>> GSoC is
>>> not unusual
>>> in my experience as a previous GSoC-mentor.
>>>
>>> The idea of working 100% of the full GSoC time
>>> from the
>>> moment it
>>> starts to the moment it finishes is not possible
>>> nor realistic.
>>> There is room for practical arrangements (but do
>>> make the GSoC
>>> deadlines!).
>>>
>>> Coding work does not officially start until 25
>>> May. While the
>>> "bonding" period starts April 27, it does not need
>>> to be
>>> full time.
>>>
>>> You should agree all this with your GSoC mentor as
>>> part of the
>>> submission process.
>>>
>>> Andy
>>>
>>>
>>>
>>> On 11/03/15 13:36, Menaka Madushanka wrote:
>>>
>>> Hi Stain,
>>>
>>> Relevant to my previous message....
>>>
>>> For examinations, I'll try my best to do
>>> something.
>>> Because, there is some possibility to
>>> negotiate with
>>> lecturers
>>> about this
>>> matter.
>>>
>>> But I'm not sure. :(
>>> Lectures are friendly and they are supporting
>>> us.
>>> I'll try.
>>>
>>> Cheers.
>>>
>>>
>>> On 11 March 2015 at 18:49, Menaka Madushanka
>>> <menaka12350@gmail.com
>>> <ma...@gmail.com> <mailto:menaka12350@gmail.com
>>> <ma...@gmail.com>>
>>> <mailto:menaka12350@gmail.com
>>> <ma...@gmail.com> <mailto:menaka12350@gmail.com
>>> <ma...@gmail.com>>__>__>
>>> wrote:
>>>
>>> Hi Stain,
>>>
>>> For time factor, my main problem is
>>> examinations in our
>>> faculty :-).
>>>
>>> For MID Exams that is a minor case but for
>>> End semester
>>> examinations,
>>> study leave 1 week and exam 1 week.
>>> Now I'm facing for Mid semester
>>> examination.
>>> Our end exam of this semester will be in
>>> May.
>>>
>>> Other than that there are no any problems
>>> for me. :-)
>>>
>>>
>>> Thank you very much.
>>> Cheers.
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Menaka Madushanka Jayawardena
>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>> University of Peradeniyaya.
>>> LinkedIn
>>> <http://lk.linkedin.com/in/____menakajayawardena
>>> <http://lk.linkedin.com/in/__menakajayawardena>
>>> <http://lk.linkedin.com/in/__menakajayawardena
>>> <http://lk.linkedin.com/in/menakajayawardena>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Menaka Madushanka Jayawardena
>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>> University of Peradeniyaya.
>>> LinkedIn <http://lk.linkedin.com/in/__menakajayawardena
>>> <http://lk.linkedin.com/in/menakajayawardena>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Menaka Madushanka Jayawardena
>>> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
>>> University of Peradeniyaya.
>>> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>>>
>>
>>
>
>
> --
> Menaka Madushanka Jayawardena
> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> University of Peradeniyaya.
> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>
--
Menaka Madushanka Jayawardena
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Andy Seaborne <an...@apache.org>.
Having exams that overlap with the early part of GSoC is not unusual in
my experience as a previous GSoC-mentor.
The idea of working 100% of the full GSoC time from the moment it starts
to the moment it finishes is not possible nor realistic. There is room
for practical arrangements (but do make the GSoC deadlines!).
Coding work does not officially start until 25 May. While the "bonding"
period starts April 27, it does not need to be full time.
You should agree all this with your GSoC mentor as part of the
submission process.
Andy
On 11/03/15 13:36, Menaka Madushanka wrote:
> Hi Stain,
>
> Relevant to my previous message....
>
> For examinations, I'll try my best to do something.
> Because, there is some possibility to negotiate with lecturers about this
> matter.
>
> But I'm not sure. :(
> Lectures are friendly and they are supporting us.
> I'll try.
>
> Cheers.
>
>
> On 11 March 2015 at 18:49, Menaka Madushanka <me...@gmail.com> wrote:
>
>> Hi Stain,
>>
>> For time factor, my main problem is examinations in our faculty :-).
>>
>> For MID Exams that is a minor case but for End semester examinations,
>> study leave 1 week and exam 1 week.
>> Now I'm facing for Mid semester examination.
>> Our end exam of this semester will be in May.
>>
>> Other than that there are no any problems for me. :-)
>>
>>
>> Thank you very much.
>> Cheers.
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Menaka Madushanka <me...@gmail.com>.
Hi Stain,
Relevant to my previous message....
For examinations, I'll try my best to do something.
Because, there is some possibility to negotiate with lecturers about this
matter.
But I'm not sure. :(
Lectures are friendly and they are supporting us.
I'll try.
Cheers.
On 11 March 2015 at 18:49, Menaka Madushanka <me...@gmail.com> wrote:
> Hi Stain,
>
> For time factor, my main problem is examinations in our faculty :-).
>
> For MID Exams that is a minor case but for End semester examinations,
> study leave 1 week and exam 1 week.
> Now I'm facing for Mid semester examination.
> Our end exam of this semester will be in May.
>
> Other than that there are no any problems for me. :-)
>
>
> Thank you very much.
> Cheers.
>
> On 11 March 2015 at 16:44, Stian Soiland-Reyes <st...@apache.org> wrote:
>
>> ... Let's try to keep the technical bit of this thread on dev@taverna
>> - but feel free to contact me personally for private matters such as
>> your commitments.
>>
>>
>> On 10 March 2015 at 19:25, Menaka Madushanka <me...@gmail.com>
>> wrote:
>>
>> > If I understood correctly,
>> > I have to implement the generalized version of the command line tools
>> by the
>> > mid evaluation
>> >
>> https://github.com/apache/incubator-taverna-language/tree/master/taverna-scufl2-examples
>> >
>> https://github.com/apache/incubator-taverna-language/tree/master/taverna-scufl2-wfdesc
>> > https://github.com/stain/ro-combine-archive
>>
>> Right - that sounds like a good plan. Once we have a prototype up and
>> running we can see better what fits well.
>>
>> This should include writing some documentation on command line
>> options, example usage etc.
>>
>>
>> > After that update ExecuteWorkflow....
>>
>> Perhaps we should flesh out a more detailed plan for ExecuteWorkflow.
>>
>> Having some kind of -verbose mode with proper logging was mentioned -
>> I can help you show how to hook into the platform to get
>> notifications. Perhaps this needs some kind of verbosity level as
>> some workflows can be very active and thus be very noisy if everything
>> is logged.
>>
>>
>> Have you got any other suggestions? For instance on simplifying the
>> command line options and perhaps moving some of these to a config
>> file? Making it a bit more unix-like perhaps.
>>
>>
>>
>> stain@biggie-utopic
>> :~/src/taverna/incubator-taverna-commandline/taverna-commandline-product/target/apache-taverna-commandline-3.1.0.incubating-SNAPSHOT-dev/apache-taverna-commandline-3.1.0.incubating-SNAPSHOT$
>> ./executeworkflow.sh --help
>> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=400m;
>> support was removed in 8.0
>> usage: executeworkflow [options] [workflow]
>> -bundle <bundle> Save outputs to a new Workflow
>> Run Bundle (zip).
>> -clientserver Connect as a client to a derby
>> server instance.
>> -cmdir <directory path> Absolute path to a directory
>> where Credential Manager's files
>> (keystore and truststore) are
>> located.
>> -cmpassword Indicate that the master password
>> for Credential Manager will be
>> provided on standard input.
>> -dbproperties <filename> Load a properties file to
>> configure the database.
>> -embedded Connect to an embedded Derby
>> database. This can prevent
>> mulitple invocations.
>> -help Display comprehensive help
>> information.
>> -inmemory Run the workflow with data stored
>> in-memory rather than in a
>> database (this is the default
>> option). This can give
>> performance inprovements, at the
>> cost of overall memory usage.
>> -inputdelimiter <inputname delimiter> Cause an inputvalue or inputfile
>> to be split into a list according
>> to the delimiter. The associated
>> workflow input must be expected
>> to receive a list.
>> -inputdoc <document> Load inputs from a Baclava
>> document.
>> -inputfile <inputname filename> Load the named input from file or
>> URL.
>> -inputvalue <inputname value> Directly use the value for the
>> named input.
>> -logfile <filename> The logfile to which more verbose
>> logging will be written to.
>> -outputdir <directory> Save outputs as files in
>> directory, default is to make a
>> new directory
>> workflowName_output.
>> -port <portnumber> The port that the database is
>> running on. If set requested to
>> start its own internal server,
>> this is the start port that will
>> be used.
>> -provenance Generate provenance information
>> and store it in the database.
>> -startdb Automatically start an internal
>> Derby database server.
>> By default, the workflow is executed using the -inmemory option, and the
>> results are written out to a directory named after the workflow name.
>>
>> If this directory already exists then a new directory is created, and
>> appended with _<n>, where n is incremented to the next available index.
>>
>> Results are written out to files named after the output port for that
>> result.
>> If a result is composed of lists, then a directory is created for the
>> output
>> port and individual list items are named after the list element index
>> (with 1
>> being the first index). The the output is the result of an error, the
>> filename
>> is appended with '.error'.
>>
>> You can provide your own output directory with the -outputdir option.
>> There
>> will be an error if the directory already exists.
>>
>> You can also record your results to a Baclava document using -outputdoc
>> option. The document will be overwritten if it already exists.
>>
>> Inputs can be provided in three ways. Both -inputfile and -inputvalue
>> options
>> can be used together; -inputdoc option must be used on its own.
>> -inputfile and
>> -inputvalue options both take two additional arguments, the name of the
>> port
>> for the input, and either a file containing the input data, or the input
>> value
>> itself respectively.
>>
>> If one of more of your workflow inputs is a list, you can create a list
>> input by using the -inputdelimiter option, which may be used with either
>> -inputfile or -inputvalue. This option takes two parameters - an input
>> name
>> and the delimiter by which to split the input into a list.
>>
>> The delimiter may be a simple character, such as a comma or a new-line
>> character, or a regular expression. The input string, or file, will then
>> be
>> converted into a list being split by the delimiter specified. Make sure to
>> put the delimiter character in quotes as it may be interpreted by the
>> shell
>> as a special character, e.g. ;.
>>
>> If a list of greater depth (i.e. a list or lists or deeper) is required
>> then
>> you will need to use the -inputdoc option. However, if you provide an
>> input
>> of lower depth to that required, then it will automatically be wrapped in
>> one
>> or more lists up to the required depth. Providing an input of greater
>> depth
>> than that required will result in an error.
>>
>> If a workflow has a high memory requirement, then it may be better to run
>> it
>> using a database to store data rather than storing it in memory, which is
>> the
>> default option. There are three options for using a database:
>>
>> -embedded option, runs with an embedded database. This is slightly faster
>> than
>> the -clientserver option (below), but has the limitation that only one
>> executeworkflow script may be executed simultaneously.
>>
>> -clientserver option allows the workflow to be executed backed by the
>> database
>> running as a server. By default a database is not started for you, but
>> may be
>> started using -startdb option.
>>
>> -startdb option starts a database. It may be used without providing a
>> workflow
>> to allow a database to be started separately, allowing multiple
>> simultaneous
>> executeworkflow script runs.
>>
>> More advanced database configurations can be specified using -dbproperties
>> option, allowing you to take full control over the database used. This
>> takes a
>> second argument, the filename of the properties file, for which the
>> following
>> example contains the default settings:
>>
>> in_memory = true
>> provenance = false
>> connector = derby
>> port = 1527
>> dialect = org.hibernate.dialect.DerbyDialect
>> start_derby = false
>> driver = org.apache.derby.jdbc.EmbeddedDriver
>> jdbcuri = jdbc:derby:t2-database;create=true;upgrade=true
>>
>> Note that when using -dbproperties together with other options, the other
>> options take precedence.
>>
>> -cmdir option lets you specify an absolute path to a directory where
>> Credential Manager's files (keystore and truststore - containing user's
>> credentials and trusted certificates for accessing secure services) are
>> stored.
>> If not specified and the workflow requires access to these files, Taverna
>> will
>> try to find them in the default location in <TAVERNA_HOME>/security
>> somewhere
>> inside user's home directory (depending on the platform).
>>
>> -cmpassword option can be used to tell Taverna to expect the password for
>> the
>> Credential Manager on standard input. If the password is not piped in,
>> Taverna
>> will prompt you for it in the terminal and block until it is entered. Do
>> not
>> enter your password in the command line! If -cmpassword option is not
>> specified
>> and -cmdir option is used, Taverna will try to find the password in a
>> special
>> file password.txt in the directory specified with -cmdir option.
>>
>>
>>
>>
>>
>>
>> --
>> Stian Soiland-Reyes
>> Apache Taverna (incubating), Apache Commons RDF (incubating)
>> http://orcid.org/0000-0001-9842-9718
>>
>
>
>
> --
> Menaka Madushanka Jayawardena
> Faculty of Engineering, <http://www.pdn.ac.lk/eng>
> University of Peradeniyaya.
> LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
>
--
Menaka Madushanka Jayawardena
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Menaka Madushanka <me...@gmail.com>.
Hi Stain,
For time factor, my main problem is examinations in our faculty :-).
For MID Exams that is a minor case but for End semester examinations, study
leave 1 week and exam 1 week.
Now I'm facing for Mid semester examination.
Our end exam of this semester will be in May.
Other than that there are no any problems for me. :-)
Thank you very much.
Cheers.
On 11 March 2015 at 16:44, Stian Soiland-Reyes <st...@apache.org> wrote:
> ... Let's try to keep the technical bit of this thread on dev@taverna
> - but feel free to contact me personally for private matters such as
> your commitments.
>
>
> On 10 March 2015 at 19:25, Menaka Madushanka <me...@gmail.com>
> wrote:
>
> > If I understood correctly,
> > I have to implement the generalized version of the command line tools by
> the
> > mid evaluation
> >
> https://github.com/apache/incubator-taverna-language/tree/master/taverna-scufl2-examples
> >
> https://github.com/apache/incubator-taverna-language/tree/master/taverna-scufl2-wfdesc
> > https://github.com/stain/ro-combine-archive
>
> Right - that sounds like a good plan. Once we have a prototype up and
> running we can see better what fits well.
>
> This should include writing some documentation on command line
> options, example usage etc.
>
>
> > After that update ExecuteWorkflow....
>
> Perhaps we should flesh out a more detailed plan for ExecuteWorkflow.
>
> Having some kind of -verbose mode with proper logging was mentioned -
> I can help you show how to hook into the platform to get
> notifications. Perhaps this needs some kind of verbosity level as
> some workflows can be very active and thus be very noisy if everything
> is logged.
>
>
> Have you got any other suggestions? For instance on simplifying the
> command line options and perhaps moving some of these to a config
> file? Making it a bit more unix-like perhaps.
>
>
>
> stain@biggie-utopic
> :~/src/taverna/incubator-taverna-commandline/taverna-commandline-product/target/apache-taverna-commandline-3.1.0.incubating-SNAPSHOT-dev/apache-taverna-commandline-3.1.0.incubating-SNAPSHOT$
> ./executeworkflow.sh --help
> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=400m;
> support was removed in 8.0
> usage: executeworkflow [options] [workflow]
> -bundle <bundle> Save outputs to a new Workflow
> Run Bundle (zip).
> -clientserver Connect as a client to a derby
> server instance.
> -cmdir <directory path> Absolute path to a directory
> where Credential Manager's files
> (keystore and truststore) are
> located.
> -cmpassword Indicate that the master password
> for Credential Manager will be
> provided on standard input.
> -dbproperties <filename> Load a properties file to
> configure the database.
> -embedded Connect to an embedded Derby
> database. This can prevent
> mulitple invocations.
> -help Display comprehensive help
> information.
> -inmemory Run the workflow with data stored
> in-memory rather than in a
> database (this is the default
> option). This can give
> performance inprovements, at the
> cost of overall memory usage.
> -inputdelimiter <inputname delimiter> Cause an inputvalue or inputfile
> to be split into a list according
> to the delimiter. The associated
> workflow input must be expected
> to receive a list.
> -inputdoc <document> Load inputs from a Baclava
> document.
> -inputfile <inputname filename> Load the named input from file or
> URL.
> -inputvalue <inputname value> Directly use the value for the
> named input.
> -logfile <filename> The logfile to which more verbose
> logging will be written to.
> -outputdir <directory> Save outputs as files in
> directory, default is to make a
> new directory
> workflowName_output.
> -port <portnumber> The port that the database is
> running on. If set requested to
> start its own internal server,
> this is the start port that will
> be used.
> -provenance Generate provenance information
> and store it in the database.
> -startdb Automatically start an internal
> Derby database server.
> By default, the workflow is executed using the -inmemory option, and the
> results are written out to a directory named after the workflow name.
>
> If this directory already exists then a new directory is created, and
> appended with _<n>, where n is incremented to the next available index.
>
> Results are written out to files named after the output port for that
> result.
> If a result is composed of lists, then a directory is created for the
> output
> port and individual list items are named after the list element index
> (with 1
> being the first index). The the output is the result of an error, the
> filename
> is appended with '.error'.
>
> You can provide your own output directory with the -outputdir option. There
> will be an error if the directory already exists.
>
> You can also record your results to a Baclava document using -outputdoc
> option. The document will be overwritten if it already exists.
>
> Inputs can be provided in three ways. Both -inputfile and -inputvalue
> options
> can be used together; -inputdoc option must be used on its own. -inputfile
> and
> -inputvalue options both take two additional arguments, the name of the
> port
> for the input, and either a file containing the input data, or the input
> value
> itself respectively.
>
> If one of more of your workflow inputs is a list, you can create a list
> input by using the -inputdelimiter option, which may be used with either
> -inputfile or -inputvalue. This option takes two parameters - an input name
> and the delimiter by which to split the input into a list.
>
> The delimiter may be a simple character, such as a comma or a new-line
> character, or a regular expression. The input string, or file, will then be
> converted into a list being split by the delimiter specified. Make sure to
> put the delimiter character in quotes as it may be interpreted by the shell
> as a special character, e.g. ;.
>
> If a list of greater depth (i.e. a list or lists or deeper) is required
> then
> you will need to use the -inputdoc option. However, if you provide an
> input
> of lower depth to that required, then it will automatically be wrapped in
> one
> or more lists up to the required depth. Providing an input of greater depth
> than that required will result in an error.
>
> If a workflow has a high memory requirement, then it may be better to run
> it
> using a database to store data rather than storing it in memory, which is
> the
> default option. There are three options for using a database:
>
> -embedded option, runs with an embedded database. This is slightly faster
> than
> the -clientserver option (below), but has the limitation that only one
> executeworkflow script may be executed simultaneously.
>
> -clientserver option allows the workflow to be executed backed by the
> database
> running as a server. By default a database is not started for you, but may
> be
> started using -startdb option.
>
> -startdb option starts a database. It may be used without providing a
> workflow
> to allow a database to be started separately, allowing multiple
> simultaneous
> executeworkflow script runs.
>
> More advanced database configurations can be specified using -dbproperties
> option, allowing you to take full control over the database used. This
> takes a
> second argument, the filename of the properties file, for which the
> following
> example contains the default settings:
>
> in_memory = true
> provenance = false
> connector = derby
> port = 1527
> dialect = org.hibernate.dialect.DerbyDialect
> start_derby = false
> driver = org.apache.derby.jdbc.EmbeddedDriver
> jdbcuri = jdbc:derby:t2-database;create=true;upgrade=true
>
> Note that when using -dbproperties together with other options, the other
> options take precedence.
>
> -cmdir option lets you specify an absolute path to a directory where
> Credential Manager's files (keystore and truststore - containing user's
> credentials and trusted certificates for accessing secure services) are
> stored.
> If not specified and the workflow requires access to these files, Taverna
> will
> try to find them in the default location in <TAVERNA_HOME>/security
> somewhere
> inside user's home directory (depending on the platform).
>
> -cmpassword option can be used to tell Taverna to expect the password for
> the
> Credential Manager on standard input. If the password is not piped in,
> Taverna
> will prompt you for it in the terminal and block until it is entered. Do
> not
> enter your password in the command line! If -cmpassword option is not
> specified
> and -cmdir option is used, Taverna will try to find the password in a
> special
> file password.txt in the directory specified with -cmdir option.
>
>
>
>
>
>
> --
> Stian Soiland-Reyes
> Apache Taverna (incubating), Apache Commons RDF (incubating)
> http://orcid.org/0000-0001-9842-9718
>
--
Menaka Madushanka Jayawardena
Faculty of Engineering, <http://www.pdn.ac.lk/eng>
University of Peradeniyaya.
LinkedIn <http://lk.linkedin.com/in/menakajayawardena>
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Stian Soiland-Reyes <st...@apache.org>.
... Let's try to keep the technical bit of this thread on dev@taverna
- but feel free to contact me personally for private matters such as
your commitments.
On 10 March 2015 at 19:25, Menaka Madushanka <me...@gmail.com> wrote:
> If I understood correctly,
> I have to implement the generalized version of the command line tools by the
> mid evaluation
> https://github.com/apache/incubator-taverna-language/tree/master/taverna-scufl2-examples
> https://github.com/apache/incubator-taverna-language/tree/master/taverna-scufl2-wfdesc
> https://github.com/stain/ro-combine-archive
Right - that sounds like a good plan. Once we have a prototype up and
running we can see better what fits well.
This should include writing some documentation on command line
options, example usage etc.
> After that update ExecuteWorkflow....
Perhaps we should flesh out a more detailed plan for ExecuteWorkflow.
Having some kind of -verbose mode with proper logging was mentioned -
I can help you show how to hook into the platform to get
notifications. Perhaps this needs some kind of verbosity level as
some workflows can be very active and thus be very noisy if everything
is logged.
Have you got any other suggestions? For instance on simplifying the
command line options and perhaps moving some of these to a config
file? Making it a bit more unix-like perhaps.
stain@biggie-utopic:~/src/taverna/incubator-taverna-commandline/taverna-commandline-product/target/apache-taverna-commandline-3.1.0.incubating-SNAPSHOT-dev/apache-taverna-commandline-3.1.0.incubating-SNAPSHOT$
./executeworkflow.sh --help
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=400m;
support was removed in 8.0
usage: executeworkflow [options] [workflow]
-bundle <bundle> Save outputs to a new Workflow
Run Bundle (zip).
-clientserver Connect as a client to a derby
server instance.
-cmdir <directory path> Absolute path to a directory
where Credential Manager's files
(keystore and truststore) are
located.
-cmpassword Indicate that the master password
for Credential Manager will be
provided on standard input.
-dbproperties <filename> Load a properties file to
configure the database.
-embedded Connect to an embedded Derby
database. This can prevent
mulitple invocations.
-help Display comprehensive help
information.
-inmemory Run the workflow with data stored
in-memory rather than in a
database (this is the default
option). This can give
performance inprovements, at the
cost of overall memory usage.
-inputdelimiter <inputname delimiter> Cause an inputvalue or inputfile
to be split into a list according
to the delimiter. The associated
workflow input must be expected
to receive a list.
-inputdoc <document> Load inputs from a Baclava
document.
-inputfile <inputname filename> Load the named input from file or
URL.
-inputvalue <inputname value> Directly use the value for the
named input.
-logfile <filename> The logfile to which more verbose
logging will be written to.
-outputdir <directory> Save outputs as files in
directory, default is to make a
new directory
workflowName_output.
-port <portnumber> The port that the database is
running on. If set requested to
start its own internal server,
this is the start port that will
be used.
-provenance Generate provenance information
and store it in the database.
-startdb Automatically start an internal
Derby database server.
By default, the workflow is executed using the -inmemory option, and the
results are written out to a directory named after the workflow name.
If this directory already exists then a new directory is created, and
appended with _<n>, where n is incremented to the next available index.
Results are written out to files named after the output port for that result.
If a result is composed of lists, then a directory is created for the output
port and individual list items are named after the list element index (with 1
being the first index). The the output is the result of an error, the filename
is appended with '.error'.
You can provide your own output directory with the -outputdir option. There
will be an error if the directory already exists.
You can also record your results to a Baclava document using -outputdoc
option. The document will be overwritten if it already exists.
Inputs can be provided in three ways. Both -inputfile and -inputvalue options
can be used together; -inputdoc option must be used on its own. -inputfile and
-inputvalue options both take two additional arguments, the name of the port
for the input, and either a file containing the input data, or the input value
itself respectively.
If one of more of your workflow inputs is a list, you can create a list
input by using the -inputdelimiter option, which may be used with either
-inputfile or -inputvalue. This option takes two parameters - an input name
and the delimiter by which to split the input into a list.
The delimiter may be a simple character, such as a comma or a new-line
character, or a regular expression. The input string, or file, will then be
converted into a list being split by the delimiter specified. Make sure to
put the delimiter character in quotes as it may be interpreted by the shell
as a special character, e.g. ;.
If a list of greater depth (i.e. a list or lists or deeper) is required then
you will need to use the -inputdoc option. However, if you provide an input
of lower depth to that required, then it will automatically be wrapped in one
or more lists up to the required depth. Providing an input of greater depth
than that required will result in an error.
If a workflow has a high memory requirement, then it may be better to run it
using a database to store data rather than storing it in memory, which is the
default option. There are three options for using a database:
-embedded option, runs with an embedded database. This is slightly faster than
the -clientserver option (below), but has the limitation that only one
executeworkflow script may be executed simultaneously.
-clientserver option allows the workflow to be executed backed by the database
running as a server. By default a database is not started for you, but may be
started using -startdb option.
-startdb option starts a database. It may be used without providing a workflow
to allow a database to be started separately, allowing multiple simultaneous
executeworkflow script runs.
More advanced database configurations can be specified using -dbproperties
option, allowing you to take full control over the database used. This takes a
second argument, the filename of the properties file, for which the following
example contains the default settings:
in_memory = true
provenance = false
connector = derby
port = 1527
dialect = org.hibernate.dialect.DerbyDialect
start_derby = false
driver = org.apache.derby.jdbc.EmbeddedDriver
jdbcuri = jdbc:derby:t2-database;create=true;upgrade=true
Note that when using -dbproperties together with other options, the other
options take precedence.
-cmdir option lets you specify an absolute path to a directory where
Credential Manager's files (keystore and truststore - containing user's
credentials and trusted certificates for accessing secure services) are stored.
If not specified and the workflow requires access to these files, Taverna will
try to find them in the default location in <TAVERNA_HOME>/security somewhere
inside user's home directory (depending on the platform).
-cmpassword option can be used to tell Taverna to expect the password for the
Credential Manager on standard input. If the password is not piped in, Taverna
will prompt you for it in the terminal and block until it is entered. Do not
enter your password in the command line! If -cmpassword option is not specified
and -cmdir option is used, Taverna will try to find the password in a special
file password.txt in the directory specified with -cmdir option.
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Stian Soiland-Reyes <st...@apache.org>.
(Menaka - moving this from private email to dev@taverna as we try to
keep everything public - hope you are OK with that.)
COMDEV-120 proposes to make a generalized version of the scufl2 and
wfdesc command line tools.
My idea was that this would be an easy way into Apache Taverna,
without requiring a full understanding of the whole architecture right
away.
The Executeworkflow command line could certainly also deserve some
updates (the -help is 5 screens long!) - specially to support some
kind of mvn-like monitoring and status output while running (e.g.
"Finished processor X, took 149s", but also some updates for Taverna 3
(which profile and/or nested workflow to execute) - this would take
you into more challenging territory with OSGi and calling the Taverna
Workflow Engine - "into the beast" so to speak. :-)
See Taverna 3 architecture diagram here:
https://www.lucidchart.com/publicSegments/view/5450cf71-3a74-4dc9-b0d4-22200a00c2c4/image.png
https://www.lucidchart.com/documents/edit/450c64b8-5166-808e-9c0b-55ba0a0057d0
(Greyed out things are desired/planned)
(BTW - Christian - where could this fit on the website?)
If you are up to the challenge, perhaps we can flesh out some
suggestion on what to update in executeworkflow as a phase 2? If the
scufl/wfdesc tool turns out to be easy, then it might not be enough
work for the whole summer.
On 9 March 2015 at 14:53, Stian Soiland-Reyes <st...@apache.org> wrote:
> Hi!
>
> Thanks for showing interest in our project proposal.
>
>
> I think the idea for the Taverna Language Command Line tool was for it
> to be separate from the existing Execute Workflow command line tool
> (the one you have looked at) - although it would be interesting to
> also consider the new functionality to be part of the command line
> tool distribution (e.g. as a separate launcher).
>
> The Taverna Language module is meant to also be used independently
> outside Taverna, and therefore has no dependencies on the rest of
> Taverna. As such it is relatively easier to package and build, and
> also it does not require the use of the OSGi plugin system that the
> executeworkflow includes.
>
> If you are looking at taverna-comandline for how we did command line
> options there - be aware that it includes a bit of trickery to work
> around OSGi threads and classloaders in passing in the command line
> arguments through the plugin system -- this should not be needed in
> the proposed Taverna Language command line as it would run with the
> regular classloader (either with a lib/*.jar collection or as a single
> executable jar.
>
>
> Note The executeworkflow program might still be slightly in a flux as
> we have been changing some of the dependencies recently, so it has not
> yet stabilized after move to Apache.
>
>
>
> On 7 March 2015 at 15:48, Menaka Madushanka <me...@gmail.com> wrote:
>> Hi,
>> I downloaded the taverna language from GIT repository and built it.
>> And I went through the references that were given in
>> https://issues.apache.org/jira/browse/COMDEV-120?filter=12330297
>>
>> So, to clarify more about the project I'd like to get some ideas about what
>> are the command line options that taverna has.
>>
>> Also I need to know whether there should be a separate command line
>> interface.
>>
>> Thank you very much.
>>
>> Cheers...!!!
>
>
>
> --
> Stian Soiland-Reyes
> Apache Taverna (incubating)
> http://orcid.org/0000-0001-9842-9718
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
Re: GSOC 2015 - Taverna: Language command line tool
Posted by Stian Soiland-Reyes <st...@apache.org>.
Hi!
Thanks for showing interest in our project proposal.
I think the idea for the Taverna Language Command Line tool was for it
to be separate from the existing Execute Workflow command line tool
(the one you have looked at) - although it would be interesting to
also consider the new functionality to be part of the command line
tool distribution (e.g. as a separate launcher).
The Taverna Language module is meant to also be used independently
outside Taverna, and therefore has no dependencies on the rest of
Taverna. As such it is relatively easier to package and build, and
also it does not require the use of the OSGi plugin system that the
executeworkflow includes.
If you are looking at taverna-comandline for how we did command line
options there - be aware that it includes a bit of trickery to work
around OSGi threads and classloaders in passing in the command line
arguments through the plugin system -- this should not be needed in
the proposed Taverna Language command line as it would run with the
regular classloader (either with a lib/*.jar collection or as a single
executable jar.
Note The executeworkflow program might still be slightly in a flux as
we have been changing some of the dependencies recently, so it has not
yet stabilized after move to Apache.
On 7 March 2015 at 15:48, Menaka Madushanka <me...@gmail.com> wrote:
> Hi,
> I downloaded the taverna language from GIT repository and built it.
> And I went through the references that were given in
> https://issues.apache.org/jira/browse/COMDEV-120?filter=12330297
>
> So, to clarify more about the project I'd like to get some ideas about what
> are the command line options that taverna has.
>
> Also I need to know whether there should be a separate command line
> interface.
>
> Thank you very much.
>
> Cheers...!!!
--
Stian Soiland-Reyes
Apache Taverna (incubating)
http://orcid.org/0000-0001-9842-9718