[jira] [Updated] (AIRFLOW-1985) Subdag tasks do not work with impersonation

["Dan Davydov (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-1985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dan Davydov updated AIRFLOW-1985:
---------------------------------
    Description: 
When using {{run_as_user}} for impersonation, airflow creates a subset of the configuration to make it available for the task during execution via {{base_task_runner.py}}. This behavior is consistent for the triggering the subdag task.
 The above code path is not executed when running tasks within a subdag using the {{SequentialExecutor}}, where each task is run direcly.

Note that in the context of subdags, tasks running for the subdags are already running in the same context of the user, so no additional impersonation is needed, but since it's not guaranteed that the user has the right configuration settings (hence why we copy a subset of the configuration during impersonation), we need to propagate those settings for the tasks within the subdag as well.

This change also requires exporting AIRFLOW_HOME and PYTHONPATH env variables in bash operator so that run_as_user can work if airflow operators are called from a bash operator (e.g. if a bash operator calls python code that imports airflow but the airflow user isn't the same as the run_as_user).

  was:
When using {{run_as_user}} for impersonation, airflow creates a subset of the configuration to make it available for the task during execution via {{base_task_runner.py}}. This behavior is consistent for the triggering the subdag task.
The above code path is not executed when running tasks within a subdag using the {{SequentialExecutor}}, where each task is run direcly.

Note that in the context of subdags, tasks running for the subdags are already running in the same context of the user, so no additional impersonation is needed, but since it's not guaranteed that the user has the right configuration settings (hence why we copy a subset of the configuration during impersonation), we need to propagate those settings for the tasks within the subdag as well.


> Subdag tasks do not work with impersonation
> -------------------------------------------
>
>                 Key: AIRFLOW-1985
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1985
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Edgar Rodriguez
>            Assignee: Edgar Rodriguez
>            Priority: Major
>
> When using {{run_as_user}} for impersonation, airflow creates a subset of the configuration to make it available for the task during execution via {{base_task_runner.py}}. This behavior is consistent for the triggering the subdag task.
>  The above code path is not executed when running tasks within a subdag using the {{SequentialExecutor}}, where each task is run direcly.
> Note that in the context of subdags, tasks running for the subdags are already running in the same context of the user, so no additional impersonation is needed, but since it's not guaranteed that the user has the right configuration settings (hence why we copy a subset of the configuration during impersonation), we need to propagate those settings for the tasks within the subdag as well.
> This change also requires exporting AIRFLOW_HOME and PYTHONPATH env variables in bash operator so that run_as_user can work if airflow operators are called from a bash operator (e.g. if a bash operator calls python code that imports airflow but the airflow user isn't the same as the run_as_user).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)