[jira] [Commented] (AIRFLOW-4030) Add Singularity Container Operator

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948742#comment-16948742 ] 

ASF GitHub Bot commented on AIRFLOW-4030:
-----------------------------------------

vsoch commented on pull request #4846: [AIRFLOW-4030] adding start to singularity for airflow
URL: https://github.com/apache/airflow/pull/4846
 
 
   Hey airflow maintainers! I'm working on an operator for [Singularity containers](https://sylabs.io/guides/3.0/user-guide/), and I have a very basic [example working](https://github.com/researchapps/airflow-example/tree/master/singularity). I'd like to do some more substantial tests beyond sleep (and I haven't written proper tests yet) but wanted to open the pull request to start discussion about tests and examples needed, etc. I'm new to airflow as of this afternoon, so thank you (in the future) for teaching me all the things!
   
   I haven't checked all the boxes below. I'm mostly afraid of Jira, so maybe if someone wants to create an issue for this, would be greatly appreciated!
   
   ### Jira
   
   - [x] My PR addresses the following [Airflow JIRA issue](https://issues.apache.org/jira/browse/AIRFLOW-4030)
   
   ### Tests
   
   - [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
   
   ### Commits
   
   - [ ] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
     1. Subject is separated from body by a blank line
     1. Subject is limited to 50 characters (not including Jira issue reference)
     1. Subject does not end with a period
     1. Subject uses the imperative mood ("add", not "adding")
     1. Body wraps at 72 characters
     1. Body explains "what" and "why", not "how"
   
   ### Documentation
   
   - [ ] In case of new functionality, my PR adds documentation that describes how to use it.
     - When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
     - All the public functions and the classes in the PR contain docstrings that explain what it does
     - If you implement backwards incompatible changes, please leave a note in the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so we can assign it to a appropriate release
   
   ### Code Quality
   
   - [ ] Passes `flake8`
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Add Singularity Container Operator
> ----------------------------------
>
>                 Key: AIRFLOW-4030
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4030
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: operators
>            Reporter: Vanessa
>            Assignee: Vanessa
>            Priority: Minor
>
> Singularity containers are comparable to Docker in the level of operation - they provide an encapsulated environment with an OS, libraries, and custom software for the user to run. The key difference is that Docker is not optimized for scientific compute because it could never be installed on a shared research cluster. Singularity, on the other hand, does not have these issues and is installed across HCP centers internationally.
> This issue is to add Singularity containers as an operator to Apache Airflow, so that we can start to explore using airflow in an HPC environment. I work with Encode DCC at Stanford, and am hopeful to explore Airflow as an alternative to the workflow manager(s) we are using. I am one of the [original Singularity developers see |https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459] that manages the Singularity Python client (spython), Singularity Hub and Singularity Registry Server, and have started working on this issue here: [https://github.com/apache/airflow/pull/4846.] Looking forward to working with you!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)