You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "t oo (Jira)" <ji...@apache.org> on 2019/08/31 05:14:00 UTC
[jira] [Updated] (AIRFLOW-5355) 1.10.4 upgrade issues - No module
named kubernetes (but i'm using localexecutor)
[ https://issues.apache.org/jira/browse/AIRFLOW-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
t oo updated AIRFLOW-5355:
--------------------------
Description:
i upgraded from 1.10.3 to 1.10.4 just now but in the main page of the ui, if i click refresh button next to my dag it shows error 'Broken DAG: [x.py] No module named kubernetes'
I have debug log enabled but have no idea why it is trying to find kubernetes.
my install steps:
pip install cryptography mysqlclient ldap3 gunicorn[gevent]
pip install kubernetes
pip install apache-airflow-1.10.4-bin.tar.gz
pip install apache-airflow-1.10.4-bin.tar.gz[kubernetes]
airflow initdb
airflow upgradedb
my only dag has:
import datetime as dt import glob import json import logging import os import subprocess import re from airflow import DAG from airflow.contrib.operators.spark_submit_operator import SparkSubmitOperator from airflow.operators.python_operator import PythonOperator from airflow.operators.dummy_operator import DummyOperator from airflow.operators.python_operator import BranchPythonOperator from airflow.hooks.base_hook import BaseHook
airflow 1.10.4, localexecutor, python2, t3.large EC2
it worked fine in 1.10.3
some other small issues:
a) tutorial.py dag is in the ui, how to remove? when i click delete it says tutorial.py not found
b) [2019-08-30 04:12:57,714] \{scheduler_job.py:924} WARNING - Tasks using non-existent pool '' will not be scheduled is in logs
c) this is in logs:
airflow-scheduler.log-[2019-08-30 09:05:38,451] \{settings.py:327} DEBUG - Failed to import airflow_local_settings.
airflow-scheduler.log-Traceback (most recent call last):
airflow-scheduler.log- File "/home/ec2-user/venv/local/lib/python2.7/site-packages/airflow/settings.py", line 315, in import_local_settings
airflow-scheduler.log- import airflow_local_settings
airflow-scheduler.log:ImportError: No module named airflow_local_settings
airflow-scheduler.log-[2019-08-30 09:05:38,452] \{logging_config.py:59} DEBUG - Unable to load custom logging, using default config instead
was:
i upgraded from 1.10.3 to 1.10.4 just now but in the main page of the ui, if i click refresh button next to my dag it shows error 'Broken DAG: [x.py] No module named kubernetes'
I have debug log enabled but have no idea why it is trying to find kubernetes.
my install steps:
pip install cryptography mysqlclient ldap3 gunicorn[gevent]
pip install kubernetes
pip install apache-airflow-1.10.4-bin.tar.gz
#cp -rv /home/ec2-user/airflow/patch/1.10.4/* /home/ec2-user/venv/lib/python2.7/site-packages/airflow/
airflow initdb
airflow upgradedb
my only dag has:
import datetime as dt import glob import json import logging import os import subprocess import re from airflow import DAG from airflow.contrib.operators.spark_submit_operator import SparkSubmitOperator from airflow.operators.python_operator import PythonOperator from airflow.operators.dummy_operator import DummyOperator from airflow.operators.python_operator import BranchPythonOperator from airflow.hooks.base_hook import BaseHook
airflow 1.10.4, localexecutor, python2, t3.large EC2
it worked fine in 1.10.3
some other small issues:
a) tutorial.py dag is in the ui, how to remove? when i click delete it says tutorial.py not found
b) [2019-08-30 04:12:57,714] \{scheduler_job.py:924} WARNING - Tasks using non-existent pool '' will not be scheduled is in logs
c) this is in logs:
airflow-scheduler.log-[2019-08-30 09:05:38,451] \{settings.py:327} DEBUG - Failed to import airflow_local_settings.
airflow-scheduler.log-Traceback (most recent call last):
airflow-scheduler.log- File "/home/ec2-user/venv/local/lib/python2.7/site-packages/airflow/settings.py", line 315, in import_local_settings
airflow-scheduler.log- import airflow_local_settings
airflow-scheduler.log:ImportError: No module named airflow_local_settings
airflow-scheduler.log-[2019-08-30 09:05:38,452] \{logging_config.py:59} DEBUG - Unable to load custom logging, using default config instead
> 1.10.4 upgrade issues - No module named kubernetes (but i'm using localexecutor)
> --------------------------------------------------------------------------------
>
> Key: AIRFLOW-5355
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5355
> Project: Apache Airflow
> Issue Type: Bug
> Components: ui
> Affects Versions: 1.10.4
> Reporter: t oo
> Priority: Major
>
> i upgraded from 1.10.3 to 1.10.4 just now but in the main page of the ui, if i click refresh button next to my dag it shows error 'Broken DAG: [x.py] No module named kubernetes'
>
> I have debug log enabled but have no idea why it is trying to find kubernetes.
>
>
> my install steps:
> pip install cryptography mysqlclient ldap3 gunicorn[gevent]
> pip install kubernetes
> pip install apache-airflow-1.10.4-bin.tar.gz
> pip install apache-airflow-1.10.4-bin.tar.gz[kubernetes]
> airflow initdb
> airflow upgradedb
>
> my only dag has:
>
> import datetime as dt import glob import json import logging import os import subprocess import re from airflow import DAG from airflow.contrib.operators.spark_submit_operator import SparkSubmitOperator from airflow.operators.python_operator import PythonOperator from airflow.operators.dummy_operator import DummyOperator from airflow.operators.python_operator import BranchPythonOperator from airflow.hooks.base_hook import BaseHook
>
>
> airflow 1.10.4, localexecutor, python2, t3.large EC2
>
> it worked fine in 1.10.3
>
>
> some other small issues:
> a) tutorial.py dag is in the ui, how to remove? when i click delete it says tutorial.py not found
> b) [2019-08-30 04:12:57,714] \{scheduler_job.py:924} WARNING - Tasks using non-existent pool '' will not be scheduled is in logs
> c) this is in logs:
> airflow-scheduler.log-[2019-08-30 09:05:38,451] \{settings.py:327} DEBUG - Failed to import airflow_local_settings.
> airflow-scheduler.log-Traceback (most recent call last):
> airflow-scheduler.log- File "/home/ec2-user/venv/local/lib/python2.7/site-packages/airflow/settings.py", line 315, in import_local_settings
> airflow-scheduler.log- import airflow_local_settings
> airflow-scheduler.log:ImportError: No module named airflow_local_settings
> airflow-scheduler.log-[2019-08-30 09:05:38,452] \{logging_config.py:59} DEBUG - Unable to load custom logging, using default config instead
--
This message was sent by Atlassian Jira
(v8.3.2#803003)