[jira] [Created] (TAVERNA-941) Use Docker microservices from workflow

["Stian Soiland-Reyes (JIRA)" <ji...@apache.org>]

Stian Soiland-Reyes created TAVERNA-941:
-------------------------------------------

             Summary: Use Docker microservices from workflow
                 Key: TAVERNA-941
                 URL: https://issues.apache.org/jira/browse/TAVERNA-941
             Project: Apache Taverna
          Issue Type: Story
          Components: Taverna Engine
            Reporter: Stian Soiland-Reyes


While TAVERNA-901 suggests a way to run Docker command line tools from Taverna, a more interesting aspect could be to run microservices from Taverna - with bindings to the dynamic ports of Docker.

This would allow usage of existing WSDL and REST services in a Taverna workflow, but with an alternative Profile to use local Docker microservices. 
This could then use server-based web UI tools like OpenRefine and Jupyter - but also pure servers like RServe, ElasticSearch and Jena Fuseki SPARQL endpoints. 

Taken to the extreme this could be to start up a local [Open PHACTS stack|https://github.com/openphacts/ops-docker/] (multiple Docker images combined with docker-compose).  (As this requires 32 GB of RAM and 0.5 TB of disk space that would not be a microservice though!)

This is more architecturally challenging than the activity of TAVERNA-901 - as it would mean preprocessing steps like

* Workflow Bundle lists required docker images
* Fetch Docker images
** Optional: load images stored in workflow bundle
* Start Docker containers -  possibly listening on dynamic ports
  * Optional: Using docker-compose for combining many containers
* Check that containers are "ready" (unsolved problem in Docker world)
* Modify/generate workflow profile to use the assigned localhost/port bindings for the docker microservices
* Run the rest of the workflow
* Shut down containers
* Remove containers (optional)
* Store provenance about Docker containers into Docker Bundle (optional)
** Snapshot docker images used into Data Bundle



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)