[jira] [Updated] (AIRFLOW-3273) Support for Passing Custom Env variables while launching k8 Pod

["Jarek Potiuk (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarek Potiuk updated AIRFLOW-3273:
----------------------------------
    Labels: gsoc gsoc2020 kubernetes mentor  (was: kubernetes)

> Support for Passing Custom Env variables while launching k8 Pod
> ---------------------------------------------------------------
>
>                 Key: AIRFLOW-3273
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3273
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: operators
>            Reporter: raman
>            Priority: Major
>              Labels: gsoc, gsoc2020, kubernetes, mentor
>
> Is there a way to provide env variables while launching K8 pod through K8 executor. we need to pass some env variable which are referred inside our Airflow Operator. so can we provide custom env variable to docker run command while launching task pod. Currently it seems that it supports predefined env variable.
> worker_configuration.py
> def _get_environment(self): """Defines any necessary environment variables for the pod executor""" env = \{ 'AIRFLOW__CORE__DAGS_FOLDER': '/tmp/dags', 'AIRFLOW__CORE__EXECUTOR': 'LocalExecutor' } if self.kube_config.airflow_configmap: env['AIRFLOW__CORE__AIRFLOW_HOME'] = self.worker_airflow_home return env
>  
> Possible solution
> At the moment there is not a way to configure environmental variables on a per-task basis, but it shouldn't be too hard to add that functionality. Extra config options can be passed through the `executor_config` on any operator:
> [https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L2423-L2437]
> Which are eventually used here to construct the kubernetes pod for the task:
> [https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/kubernetes/worker_configuration.py#L186]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)