You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "David Lum (JIRA)" <ji...@apache.org> on 2019/01/30 21:25:00 UTC
[jira] [Updated] (AIRFLOW-3258) Add ability to set Environment
Variables for pods brought up by Kubernetes Executor
[ https://issues.apache.org/jira/browse/AIRFLOW-3258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Lum updated AIRFLOW-3258:
-------------------------------
Description: Currently the only way to add arbitrary environment variables to pods brought up by the KubernetesExecutor is to add them using Kubernetes secrets, and then specifying those secrets in the in the kubernetes_secrets section of the the airflow.cfg. However, not all environment variables are secrets and need to be encoded. The suggestion would be to add another section to the airflow.cfg, kubernetes_environment_variables. On top of this, a short hand could be to add all the env vars in the current environment to the pods being brought up. This could be set with a boolean flag in the kubernetes section of the airflow.cfg which would to default to false, and have a name such as insert_current_environment. Any environment variables set in the kubernetes_environment_variables section, as well as those that are hard coded such as AIRFLOW_\_CORE\__EXECUTOR would overwrite those present in the current environment when applied to the worker pod. (was: Currently the only way to add arbitrary environment variables to pods brought up by the KubernetesExecutor is to add them using Kubernetes secrets, and then specifying those secrets in the in the kubernetes_secrets section of the the airflow.cfg. However, not all environment variables are secrets and need to be encoded. The suggestion would be to add another section to the airflow.cfg, kubernetes_environment_variables. On top of this, a short hand could be to add all the env vars in the current environment to the pods being brought up. This could be set with a boolean flag in the kubernetes section of the airflow.cfg which would to default to false, and have a name such as insert_current_environment. Any environment variables set in the kubernetes_environment_variables section, as well as those that are hard coded such as AIRFLOW__CORE____EXECUTOR would overwrite those present in the current environment when applied to the worker pod.)
> Add ability to set Environment Variables for pods brought up by Kubernetes Executor
> -----------------------------------------------------------------------------------
>
> Key: AIRFLOW-3258
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3258
> Project: Apache Airflow
> Issue Type: Improvement
> Components: kubernetes
> Affects Versions: 1.10.0
> Reporter: David Lum
> Assignee: David Lum
> Priority: Minor
> Labels: kubernetes
> Fix For: 1.10.0
>
>
> Currently the only way to add arbitrary environment variables to pods brought up by the KubernetesExecutor is to add them using Kubernetes secrets, and then specifying those secrets in the in the kubernetes_secrets section of the the airflow.cfg. However, not all environment variables are secrets and need to be encoded. The suggestion would be to add another section to the airflow.cfg, kubernetes_environment_variables. On top of this, a short hand could be to add all the env vars in the current environment to the pods being brought up. This could be set with a boolean flag in the kubernetes section of the airflow.cfg which would to default to false, and have a name such as insert_current_environment. Any environment variables set in the kubernetes_environment_variables section, as well as those that are hard coded such as AIRFLOW_\_CORE\__EXECUTOR would overwrite those present in the current environment when applied to the worker pod.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)