[jira] [Commented] (AIRFLOW-3289) BashOperator mangles {{\}} escapes in commands

["Daniel Lamblin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859815#comment-16859815 ] 

Daniel Lamblin commented on AIRFLOW-3289:
-----------------------------------------

Your comment makes it see like perhaps jinja templating is changing {{\\}} to {{\}} but it doesn't happen with plain jinja2, so I'm not sure where in the code this would happen…
{code:java}
Python 3.6.2 (default, Jul 17 2017, 16:44:45)
...
>>> import jinja2
>>> e=jinja2.Environment()
>>> t=e.from_string('''
... cd $HOME/issue_example
... cat example.csv | sed 's;,,;,\\N,;g' > example_processed.csv
... ''')
>>> t.render()
"\ncd $HOME/issue_example\ncat example.csv | sed 's;,,;,\\N,;g' > example_processed.csv"{code}
The difference with using an sh file is odd because the rendered template is written out to a file in both cases before execution: [https://github.com/apache/airflow/blob/1.10.0/airflow/operators/bash_operator.py#L81]

 

> BashOperator mangles {{\}} escapes in commands
> ----------------------------------------------
>
>                 Key: AIRFLOW-3289
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3289
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: 1.10.0
>            Reporter: Nikolay Semyachkin
>            Priority: Major
>         Attachments: example.csv, issue_proof.py
>
>
> I want to call a sed command on csv file to replace empty values (,,) with \N.
> I can do it with the following bash command 
> {code:java}
> cat example.csv | sed 's;,,;,\\N,;g' > example_processed.csv{code}
> But when I try to do the same with airflow BashOperator, it substitutes ,, with N (instead of \N).
>  
> I attached the code and csv file to reproduce.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)