You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Vanessa (JIRA)" <ji...@apache.org> on 2019/03/06 17:32:00 UTC

[jira] [Created] (AIRFLOW-4030) Add Singularity Container Operator

Vanessa created AIRFLOW-4030:
--------------------------------

             Summary: Add Singularity Container Operator
                 Key: AIRFLOW-4030
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4030
             Project: Apache Airflow
          Issue Type: Improvement
          Components: operators
            Reporter: Vanessa
            Assignee: Vanessa


Singularity containers are comparable to Docker in the level of operation - they provide an encapsulated environment with an OS, libraries, and custom software for the user to run. The key difference is that Docker is not optimized for scientific compute because it could never be installed on a shared research cluster. Singularity, on the other hand, does not have these issues and is installed across HCP centers internationally.

This issue is to add Singularity containers as an operator to Apache Airflow, so that we can start to explore using airflow in an HPC environment. I work with Encode DCC at Stanford, and am hopeful to explore Airflow as an alternative to the workflow manager(s) we are using. I am one of the [original Singularity developers see |https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459] that manages the Singularity Python client (spython), Singularity Hub and Singularity Registry Server, and have started working on this issue here: [https://github.com/apache/airflow/pull/4846.] Looking forward to working with you!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)