[jira] [Commented] (AIRFLOW-1409) SparkSubmitHook doesn't generate a valid master URL

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16731475#comment-16731475 ] 

ASF GitHub Bot commented on AIRFLOW-1409:
-----------------------------------------

stale[bot] commented on pull request #2451: [AIRFLOW-1409] Parse ENV connections differently in SparkSubmitHook
URL: https://github.com/apache/incubator-airflow/pull/2451
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> SparkSubmitHook doesn't generate a valid master URL
> ---------------------------------------------------
>
>                 Key: AIRFLOW-1409
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1409
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Luis Angel Vicente Sanchez
>            Priority: Major
>
> If you configure the connection using an ENV variable (and possibly through the UI), SparkSubmitHook is ignoring the scheme and doesn't produce valid master URL. Example, this ENV variable,
> AIRFLOW_CONN_SPARK_CLUSTER=spark://spark-master:7077 
> will produce the following master URL:
> spark-master:7077
> This URL is not valid and the job submission fails. The issue is how the url is parsed to create a connection, I have a partial fix working and I will submit a PR to github soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)