You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "t oo (Jira)" <ji...@apache.org> on 2019/12/26 17:20:00 UTC
[jira] [Updated] (AIRFLOW-5406) Allow spark_submit_hook to run
without pip install kubernetes/airflow[kubernetes]
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
t oo updated AIRFLOW-5406:
--------------------------
Description:
Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]
k8s dependencies causing issues for non-k8s users based on 1.10.6:
*settings.py:*
from airflow.contrib.kubernetes.pod import Pod
def pod_mutation_hook(pod):
*models/__init__.py*
from airflow.models.kubernetes import KubeWorkerIdentifier, KubeResourceVersion
*executors/__init__.py*
KubernetesExecutor = "KubernetesExecutor"
elif executor_name == Executors.KubernetesExecutor:
from airflow.contrib.executors.kubernetes_executor import KubernetesExecutor
return KubernetesExecutor()
*spark_submit_hook.py*
from airflow.contrib.kubernetes import kube_client
was:Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]
> Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]
> ---------------------------------------------------------------------------------
>
> Key: AIRFLOW-5406
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5406
> Project: Apache Airflow
> Issue Type: Improvement
> Components: dependencies, executor-kubernetes, executors
> Affects Versions: 1.10.5
> Reporter: t oo
> Assignee: Daniel Imberman
> Priority: Minor
>
> Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]
> k8s dependencies causing issues for non-k8s users based on 1.10.6:
> *settings.py:*
> from airflow.contrib.kubernetes.pod import Pod
> def pod_mutation_hook(pod):
> *models/__init__.py*
> from airflow.models.kubernetes import KubeWorkerIdentifier, KubeResourceVersion
> *executors/__init__.py*
> KubernetesExecutor = "KubernetesExecutor"
> elif executor_name == Executors.KubernetesExecutor:
> from airflow.contrib.executors.kubernetes_executor import KubernetesExecutor
> return KubernetesExecutor()
> *spark_submit_hook.py*
> from airflow.contrib.kubernetes import kube_client
--
This message was sent by Atlassian Jira
(v8.3.4#803005)