Design and Implementation of Docker Activity Plugin

[Nadeesh Dilanga <na...@gmail.com>]

Hi all,
@"Finalize Docker Invoke JSON format" we are finalizing the JSON for Docker
start and what input params required from workflow.

We decide to do some research on existing activity plugins as a starting
point on designing Docker Activity plugin. Initial suggestion was to check
out external-tool-activity or command line tool.

But given we can start Docker through REST, I had a look at the code of
taverna-rest-activity/RESTActivity.java. Seems we can reuse
org.apache.taverna.activities.rest.HTTPRequestHandler class and invoke
docker APIs.

RESTActivity will put the entire http response to the callback as follows.
So may be we might be able to extend this to achieve our requirement.  What
do you think about this ? Any suggestions ?

callback.receiveResult(outputs, new int[0]);





<https://github.com/apache/incubator-taverna-common-activities/tree/master/taverna-rest-activity>

Re: Design and Implementation of Docker Activity Plugin

[Nadeesh Dilanga <na...@gmail.com>]

Hi Alan,
Yes that's what I wanted to confirm. And big +1 for directly using Apache
HttpClient without over engineering. So I will take it as a +1 to starting
docker through REST from activity plugin.

I will assume I will get all required parameters to invoke the REST
API(what get finalized in the JSON format mail thread) from the
configuration bean an start writing an activity plugin. Please let me know
if there is any other concerns.


On Thu, Jun 9, 2016 at 7:30 AM, Alan Williams <al...@googlemail.com>
wrote:

> On 09-Jun-16 12:22, Nadeesh Dilanga wrote:
>
>> Hi Stian, Hi Alan,
>> Do you think we shouldn't use REST invocations to start Docker, but better
>> to use something else ?
>>
>
> Do you mean invocations of the REST activity? I think it would be better
> to use the Apache HttpClient directly. That will make it easier to debug
> and to maintain.
>
> If you are asking if it is best to talk to docker using the Docker Remote
> API or by executing command line calls, I think that the API is the better
> way to proceed.
>
>
> (Java API/use Taverna Command Line tool to execute
>> direct docker commands/anything else).
>>
>
> Alan
>
>

Re: Design and Implementation of Docker Activity Plugin

[Alan Williams <al...@googlemail.com>]

On 09-Jun-16 12:22, Nadeesh Dilanga wrote:
> Hi Stian, Hi Alan,
> Do you think we shouldn't use REST invocations to start Docker, but better
> to use something else ?

Do you mean invocations of the REST activity? I think it would be better 
to use the Apache HttpClient directly. That will make it easier to debug 
and to maintain.

If you are asking if it is best to talk to docker using the Docker 
Remote API or by executing command line calls, I think that the API is 
the better way to proceed.

> (Java API/use Taverna Command Line tool to execute
> direct docker commands/anything else).

Alan

Re: Design and Implementation of Docker Activity Plugin

[Nadeesh Dilanga <na...@gmail.com>]

Hi Stian, Hi Alan,
Do you think we shouldn't use REST invocations to start Docker, but better
to use something else ? (Java API/use Taverna Command Line tool to execute
direct docker commands/anything else).

On Wed, Jun 8, 2016 at 12:20 PM, Stian Soiland-Reyes <st...@apache.org>
wrote:

> I think this is good investigation. I think prototyping with the REST
> activity manually from the Taverna Workbench 2.5 is a good way to
> learn about the Docker API. Perhaps you could share those experiments
> on say myExperiment (if you have workflows) or in the Taverna wiki?
>
> However I am not sure if the REST activity will work well within the
> workflow for end-users, if the REST API calls to the Docker API gets a
> bit detailed or have to sent with POST - as the user would then have
> to create the JSON manually in a previous workflow step.
>
> Also for the outputs, I would think a Docker Activity would output the
> STDOUT/STDERR from the docker container, and perhaps a selection of
> files from a configured /volume?  But perhaps I'm thinking too much
> like the current Tool activity!
>
>
> On 8 June 2016 at 04:44, Nadeesh Dilanga <na...@gmail.com> wrote:
> > Hi all,
> > @"Finalize Docker Invoke JSON format" we are finalizing the JSON for
> Docker
> > start and what input params required from workflow.
> >
> > We decide to do some research on existing activity plugins as a starting
> > point on designing Docker Activity plugin. Initial suggestion was to
> check
> > out external-tool-activity or command line tool.
> >
> > But given we can start Docker through REST, I had a look at the code of
> > taverna-rest-activity/RESTActivity.java. Seems we can reuse
> > org.apache.taverna.activities.rest.HTTPRequestHandler class and invoke
> > docker APIs.
> >
> > RESTActivity will put the entire http response to the callback as
> follows.
> > So may be we might be able to extend this to achieve our requirement.
> What
> > do you think about this ? Any suggestions ?
> >
> > callback.receiveResult(outputs, new int[0]);
> >
> >
> >
> >
> >
> > <
> https://github.com/apache/incubator-taverna-common-activities/tree/master/taverna-rest-activity
> >
>
>
>
> --
> Stian Soiland-Reyes
> Apache Taverna (incubating), Apache Commons
> http://orcid.org/0000-0001-9842-9718
>

Re: Design and Implementation of Docker Activity Plugin

[Stian Soiland-Reyes <st...@apache.org>]

I think this is good investigation. I think prototyping with the REST
activity manually from the Taverna Workbench 2.5 is a good way to
learn about the Docker API. Perhaps you could share those experiments
on say myExperiment (if you have workflows) or in the Taverna wiki?

However I am not sure if the REST activity will work well within the
workflow for end-users, if the REST API calls to the Docker API gets a
bit detailed or have to sent with POST - as the user would then have
to create the JSON manually in a previous workflow step.

Also for the outputs, I would think a Docker Activity would output the
STDOUT/STDERR from the docker container, and perhaps a selection of
files from a configured /volume?  But perhaps I'm thinking too much
like the current Tool activity!


On 8 June 2016 at 04:44, Nadeesh Dilanga <na...@gmail.com> wrote:
> Hi all,
> @"Finalize Docker Invoke JSON format" we are finalizing the JSON for Docker
> start and what input params required from workflow.
>
> We decide to do some research on existing activity plugins as a starting
> point on designing Docker Activity plugin. Initial suggestion was to check
> out external-tool-activity or command line tool.
>
> But given we can start Docker through REST, I had a look at the code of
> taverna-rest-activity/RESTActivity.java. Seems we can reuse
> org.apache.taverna.activities.rest.HTTPRequestHandler class and invoke
> docker APIs.
>
> RESTActivity will put the entire http response to the callback as follows.
> So may be we might be able to extend this to achieve our requirement.  What
> do you think about this ? Any suggestions ?
>
> callback.receiveResult(outputs, new int[0]);
>
>
>
>
>
> <https://github.com/apache/incubator-taverna-common-activities/tree/master/taverna-rest-activity>



-- 
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons
http://orcid.org/0000-0001-9842-9718