[GitHub] [airflow] sakshi-bansal commented on pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

[GitBox <gi...@apache.org>]

sakshi-bansal commented on pull request #7163:
URL: https://github.com/apache/airflow/pull/7163#issuecomment-739774827


   Airflow version: 1.10.13
   
   Trying to deploy https://github.com/apache/airflow/blob/master/airflow/providers/cncf/kubernetes/example_dags/example_spark_kubernetes.py. Getting the following error **No module named 'airflow.providers'**. Executed **pip install apache-airflow-backport-providers-cncf-kubernetes** from the airflow worker/web/scheduler pod I can now import the `from airflow.providers.cncf.kubernetes.operators.spark_kubernetes import SparkKubernetesOperator`  . Getting the following error :
   Traceback (most recent call last):
     File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/dagbag.py", line 256, in process_file
       m = imp.load_source(mod_name, filepath)
     File "/usr/local/lib/python3.6/imp.py", line 172, in load_source
       module = _load(spec)
     File "<frozen importlib._bootstrap>", line 684, in _load
     File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
     File "<frozen importlib._bootstrap_external>", line 678, in exec_module
     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
     File "/opt/airflow/dags/sparkoperator.py", line 4, in <module>
       from airflow.providers.cncf.kubernetes.operators.spark_kubernetes import SparkKubernetesOperator
   ModuleNotFoundError: No module named 'airflow.providers'
   /home/airflow/.local/lib/python3.6/site-packages/airflow/kubernetes/pod_generator.py:39: DeprecationWarning: This module is deprecated. Please use `airflow.kubernetes.pod`.
     from airflow.contrib.kubernetes.pod import _extract_volume_mounts 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org