You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by ra...@gmail.com,
ra...@gmail.com on 2018/10/29 15:16:13 UTC
Passing Custom Env variables while launching k8 Pod
Hi All,
Is there a way to provide env variables while launching K8 pod through K8 executor.
we need to pass some env variable which are referred inside our Airflow Operator.
so can we provide custom env variable to docker run command while launching task pod.
Currently it seems that it supports predefined env variable.
worker_configuration.py
def _get_environment(self):
"""Defines any necessary environment variables for the pod executor"""
env = {
'AIRFLOW__CORE__DAGS_FOLDER': '/tmp/dags',
'AIRFLOW__CORE__EXECUTOR': 'LocalExecutor'
}
if self.kube_config.airflow_configmap:
env['AIRFLOW__CORE__AIRFLOW_HOME'] = self.worker_airflow_home
return env
Thanks,
Raman Gupta
Re: Passing Custom Env variables while launching k8 Pod
Posted by Grant Nicholas <gr...@u.northwestern.edu>.
At the moment there is not a way to configure environmental variables on a
per-task basis, but it shouldn't be too hard to add that functionality.
Extra config options can be passed through the `executor_config` on any
operator:
https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L2423-L2437
Which are eventually used here to construct the kubernetes pod for the task:
https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/kubernetes/worker_configuration.py#L186
On Mon, Oct 29, 2018 at 10:16 AM ramandumcs@gmail.com <ra...@gmail.com>
wrote:
> Hi All,
> Is there a way to provide env variables while launching K8 pod through K8
> executor.
> we need to pass some env variable which are referred inside our Airflow
> Operator.
> so can we provide custom env variable to docker run command while
> launching task pod.
> Currently it seems that it supports predefined env variable.
> worker_configuration.py
> def _get_environment(self):
> """Defines any necessary environment variables for the pod
> executor"""
> env = {
> 'AIRFLOW__CORE__DAGS_FOLDER': '/tmp/dags',
> 'AIRFLOW__CORE__EXECUTOR': 'LocalExecutor'
> }
> if self.kube_config.airflow_configmap:
> env['AIRFLOW__CORE__AIRFLOW_HOME'] = self.worker_airflow_home
> return env
>
> Thanks,
> Raman Gupta
>