[jira] [Resolved] (AIRFLOW-4799) tests using bash operator fail flakily because jinja2 rendering of environment variables

["Jarek Potiuk (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarek Potiuk resolved AIRFLOW-4799.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: 2.0.0
                   1.10.4

> tests using bash operator fail flakily because jinja2 rendering of environment variables
> ----------------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-4799
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4799
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 2.0.0
>            Reporter: Daniel Standish
>            Assignee: Daniel Standish
>            Priority: Trivial
>             Fix For: 1.10.4, 2.0.0
>
>
> In test_retry_delay in task instance tests we see this:
> {code:python}
>         ti = TI(
>             task=task, execution_date=timezone.utcnow())
>         self.assertEqual(ti.try_number, 1)
>         # first run -- up for retry
>         run_with_error(ti)
>         self.assertEqual(ti.state, State.UP_FOR_RETRY)
>         self.assertEqual(ti.try_number, 2)
>         # second run -- still up for retry because retry_delay hasn't expired
>         run_with_error(ti)
>         self.assertEqual(ti.state, State.UP_FOR_RETRY)
>         # third run -- failed
>         time.sleep(3)
>         run_with_error(ti)
>         self.assertEqual(ti.state, State.FAILED)
> {code}
> The same TI is re-run multiple times.
> The problem is that in the execute method of BashOperator, the {{env}} attribute is modified, and updated to include a copy of the parent bash env.  Then when this TI is executed another time, it again tries to render {{env}}, because it is a templated parameter.  And it will attempt to load every {{.sh}} or {{.bash}} file found in `env` as a template.  If one file does not exist, test will fail with template not found error.
> The modifications of env in execute method should be local to execute method. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)