You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Niranda Perera <ni...@cse.mrt.ac.lk> on 2017/10/02 07:43:01 UTC
Mesos Marathon support in Airflow
Hi all,
I know Airflow currently supports Mesos executor, but may I know if it
supports Mesos Marathon? The idea is to use Airflow to run task as a
container orchestrator.
On a side note, AF only supports a single executor by design. So, if at all
someone integrated Mesos Marathon with AF, it should be done using an
operator, isn't it?
Best regards
Niranda Perera
Research Assistant
Dept of CSE, University of Moratuwa
niranda.17@cse.mrt.ac.lk
+94 71 554 8430
https://lk.linkedin.com/in/niranda
Re: Mesos Marathon support in Airflow
Posted by Andrew Phillips <an...@apache.org>.
Hi Niranda
> I know Airflow currently supports Mesos executor, but may I know if it
> supports Mesos Marathon? The idea is to use Airflow to run task as a
> container orchestrator.
Could you explain in a bit more detail what you mean by "Airflow
supporting Marathon"? Airflow certainly runs *inside* Marathon happily
enough (that's how we're running it in prod), but from what you write
I'm assuming that you're thinking about Airflow creating and managing
*other* Marathon applications?
Given that Marathon manages long-running, non-scheduled tasks and
Airflow seems aimed more at running terminating sequences of tasks on a
schedule, where do you see the potential overlap/integration
opportunity?
Regards
ap