[jira] [Commented] (AIRFLOW-5633) Apache Airflow Integration with HashiCorp Nomad

["Ash Berlin-Taylor (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16949262#comment-16949262 ] 

Ash Berlin-Taylor commented on AIRFLOW-5633:
--------------------------------------------

We'd love to accept a nicer integration with Nomad, so long as the unit tests are comprehensive enough, and we have the means of testing it in a real nomad environment.

> Apache Airflow Integration with HashiCorp Nomad
> -----------------------------------------------
>
>                 Key: AIRFLOW-5633
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5633
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: executors, operators
>    Affects Versions: 1.10.5
>            Reporter: Yishan
>            Priority: Critical
>
> Nomad is an orchestrator, similar to Kubernetes and used in production by companies like PagerDuty, Target, Citadel, Trivago, Pandora, Roblox, etc.  Built by HashiCorp (company behind other tools such as Terraform, Vault, and Consul).
> We've gotten many requests from our customers and open source community for an integration with Airflow.  There are a variety of community PRs from supportive members who are trying to build their own (linked notable ones below) - our approach would be to build one officially supported integration ourselves for all our users and customers.  
> [https://github.com/apache/airflow/pull/2708]
> [https://github.com/hashicorp/nomad/issues/419]
> We'd love to understand how much interest and support we can gauge from the Airflow committee around an integration like this.  Ideally, we'd aim to avoid a situation where our integration is not accepted + Nomad users using Airflow are subject to an incomplete/less-than-ideal experience on a lagging HashiCorp-maintained-fork. 
> This has unfortunately happened with our Spark integration, so we'd love to gauge the Airflow committee early-on in the process before we potentially build the integration.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)