You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/09/03 17:54:00 UTC
[jira] [Commented] (AIRFLOW-4030) Add Singularity Container
Operator
[ https://issues.apache.org/jira/browse/AIRFLOW-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921609#comment-16921609 ]
ASF GitHub Bot commented on AIRFLOW-4030:
-----------------------------------------
vsoch commented on pull request #4846: [AIRFLOW-4030] adding start to singularity for airflow
URL: https://github.com/apache/airflow/pull/4846
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Add Singularity Container Operator
> ----------------------------------
>
> Key: AIRFLOW-4030
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4030
> Project: Apache Airflow
> Issue Type: Improvement
> Components: operators
> Reporter: Vanessa
> Assignee: Vanessa
> Priority: Minor
>
> Singularity containers are comparable to Docker in the level of operation - they provide an encapsulated environment with an OS, libraries, and custom software for the user to run. The key difference is that Docker is not optimized for scientific compute because it could never be installed on a shared research cluster. Singularity, on the other hand, does not have these issues and is installed across HCP centers internationally.
> This issue is to add Singularity containers as an operator to Apache Airflow, so that we can start to explore using airflow in an HPC environment. I work with Encode DCC at Stanford, and am hopeful to explore Airflow as an alternative to the workflow manager(s) we are using. I am one of the [original Singularity developers see |https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459] that manages the Singularity Python client (spython), Singularity Hub and Singularity Registry Server, and have started working on this issue here: [https://github.com/apache/airflow/pull/4846.] Looking forward to working with you!
--
This message was sent by Atlassian Jira
(v8.3.2#803003)