You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by ra...@gmail.com, ra...@gmail.com on 2018/10/29 15:16:13 UTC

Passing Custom Env variables while launching k8 Pod

Hi All, 
Is there a way to provide  env variables while launching K8 pod through K8 executor.
we need to pass some env variable which are referred inside our Airflow Operator.
so can we provide custom env variable to docker run command while launching task pod.
Currently it seems that it supports predefined env variable.
worker_configuration.py
    def _get_environment(self):
        """Defines any necessary environment variables for the pod executor"""
        env = {
            'AIRFLOW__CORE__DAGS_FOLDER': '/tmp/dags',
            'AIRFLOW__CORE__EXECUTOR': 'LocalExecutor'
        }
        if self.kube_config.airflow_configmap:
            env['AIRFLOW__CORE__AIRFLOW_HOME'] = self.worker_airflow_home
        return env

Thanks,
Raman Gupta

Re: Passing Custom Env variables while launching k8 Pod

Posted by Grant Nicholas <gr...@u.northwestern.edu>.
At the moment there is not a way to configure environmental variables on a
per-task basis, but it shouldn't be too hard to add that functionality.

Extra config options can be passed through the `executor_config` on any
operator:
https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L2423-L2437

Which are eventually used here to construct the kubernetes pod for the task:
https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/kubernetes/worker_configuration.py#L186


On Mon, Oct 29, 2018 at 10:16 AM ramandumcs@gmail.com <ra...@gmail.com>
wrote:

> Hi All,
> Is there a way to provide  env variables while launching K8 pod through K8
> executor.
> we need to pass some env variable which are referred inside our Airflow
> Operator.
> so can we provide custom env variable to docker run command while
> launching task pod.
> Currently it seems that it supports predefined env variable.
> worker_configuration.py
>     def _get_environment(self):
>         """Defines any necessary environment variables for the pod
> executor"""
>         env = {
>             'AIRFLOW__CORE__DAGS_FOLDER': '/tmp/dags',
>             'AIRFLOW__CORE__EXECUTOR': 'LocalExecutor'
>         }
>         if self.kube_config.airflow_configmap:
>             env['AIRFLOW__CORE__AIRFLOW_HOME'] = self.worker_airflow_home
>         return env
>
> Thanks,
> Raman Gupta
>