You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Bolke de Bruin (JIRA)" <ji...@apache.org> on 2017/07/12 09:29:00 UTC
[jira] [Resolved] (AIRFLOW-1326) Contrib Spark Submit Hook should
be able to produce arguments containing spaces.
[ https://issues.apache.org/jira/browse/AIRFLOW-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bolke de Bruin resolved AIRFLOW-1326.
-------------------------------------
Resolution: Fixed
Fix Version/s: (was: Airflow 1.8)
(was: Airflow 2.0)
1.8.3
Issue resolved by pull request #2382
[https://github.com/apache/incubator-airflow/pull/2382]
> Contrib Spark Submit Hook should be able to produce arguments containing spaces.
> --------------------------------------------------------------------------------
>
> Key: AIRFLOW-1326
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1326
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib, hooks
> Affects Versions: Airflow 2.0, Airflow 1.8
> Reporter: Ash Berlin-Taylor
> Assignee: Vianney FOUCAULT
> Fix For: 1.8.3
>
>
> AIRFLOW-1184 introduced a bug making it impossible to include an argument with an embedded space in the SparkSubmitOperator. (I believe the original bug report should have been closed as wontfix/not-a-bug as it was user error in how the SparkSubmitOperator is being called)
> i.e. the correct way to achieve what was wanted in the bug is like this:
> {code}
> 'application_args': [
> '-f', ' foo',
> '--bar', 'bar',
> '--start', '{{ macros.ds_add(ds, -1)}}',
> '--end', '{{ ds }}'
> ]
> {code}
> This was possibly compounded as the tests were showing something that would never "work"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)