You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Kyle Hamlin (JIRA)" <ji...@apache.org> on 2018/08/09 16:45:00 UTC

[jira] [Closed] (AIRFLOW-2079) Add mesos polling to spark_submit_hook

     [ https://issues.apache.org/jira/browse/AIRFLOW-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kyle Hamlin closed AIRFLOW-2079.
--------------------------------
    Resolution: Later

> Add mesos polling to spark_submit_hook
> --------------------------------------
>
>                 Key: AIRFLOW-2079
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2079
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib, hooks
>    Affects Versions: 1.10.0
>            Reporter: Kyle Hamlin
>            Assignee: Kyle Hamlin
>            Priority: Minor
>              Labels: easyfix
>             Fix For: 1.10.0
>
>
> The SparkSubmitHook has a method `_resolve_should_track_driver_status` which tells the hook whether or not to poll the spark-submit driver for its status. Currently, the method only checks for a master containing `spark://` in order to poll the driver. Since Mesos is an available master I think we should also include `mesos://` in this check.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)