You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org> on 2019/05/18 02:26:00 UTC
[jira] [Updated] (AIRFLOW-3372) Unable to start airflow scheduler
[ https://issues.apache.org/jira/browse/AIRFLOW-3372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aizhamal Nurmamat kyzy updated AIRFLOW-3372:
--------------------------------------------
Component/s: (was: kubernetes)
(was: docker)
> Unable to start airflow scheduler
> ---------------------------------
>
> Key: AIRFLOW-3372
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3372
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 1.9.0
> Environment: Kubernetes,docker
> Reporter: MADHANKUMAR C
> Priority: Blocker
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> *I have installed airflow in kubernetes cluster.When i am installing airflow ,i am unable to start the scheduler.The below is the log of scheduler container.*
> [2018-11-20 12:02:40,860] {{__init__.py:51}} INFO - Using executor SequentialExecutor
> [2018-11-20 12:02:40,973] {{cli_action_loggers.py:69}} ERROR - Failed on pre-execution callback using <function default_action_log at 0x7f26b730b620>
> Traceback (most recent call last):
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
> context)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
> cursor.execute(statement, parameters)
> sqlite3.OperationalError: no such table: log
> The above exception was the direct cause of the following exception:
> Traceback (most recent call last):
> File "/usr/local/lib/python3.5/dist-packages/airflow/utils/cli_action_loggers.py", line 67, in on_pre_execution
> cb(**kwargs)
> File "/usr/local/lib/python3.5/dist-packages/airflow/utils/cli_action_loggers.py", line 99, in default_action_log
> session.commit()
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 927, in commit
> self.transaction.commit()
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 467, in commit
> self._prepare_impl()
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 447, in _prepare_impl
> self.session.flush()
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2209, in flush
> self._flush(objects)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2329, in _flush
> transaction.rollback(_capture_exception=True)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
> compat.reraise(exc_type, exc_value, exc_tb)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 187, in reraise
> raise value
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2293, in _flush
> flush_context.execute()
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
> rec.execute(self)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
> uow
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
> mapper, table, insert)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/persistence.py", line 835, in _emit_insert_statements
> execute(statement, params)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
> return meth(self, multiparams, params)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
> compiled_sql, distilled_params
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
> context)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
> exc_info
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 186, in reraise
> raise value.with_traceback(tb)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
> context)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
> cursor.execute(statement, parameters)
> sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: log [SQL: 'INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (?, ?, ?, ?, ?, ?, ?)'] [parameters: ('2018-11-20 12:02:40.969353', None, None, 'cli_scheduler', None, 'airflow', '
> {"host_name": "airflow-airflow-scheduler-5b5f8b9549-89dmn", "full_command": "[\'/usr/local/bin/airflow\', \'scheduler\', \'-n\', \'-1\', \'-p\']"}
> ')]
> ____________ _____________
> ____ |__( )_________ __/__ /________ __
> ____ /| |_ /__ ___/_ /_ __ /_ __ _ | /| / /
> ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
> _/_/ |_/_/ /_/ /_/ /_/ ____/____/|__/
> [2018-11-20 12:02:40,977] {{jobs.py:580}} *{color:#FF0000}ERROR - Cannot use more than 1 thread when using sqlite. Setting max_threads to 1{color}*
> /usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py:513: SAWarning: Session's state has been changed on a non-active transaction - this state will be discarded.
> "Session's state has been changed on "
> Traceback (most recent call last):
> File "/usr/local/bin/airflow", line 32, in <module>
> args.func(args)
> File "/usr/local/lib/python3.5/dist-packages/airflow/utils/cli.py", line 74, in wrapper
> return f(*args, **kwargs)
> File "/usr/local/lib/python3.5/dist-packages/airflow/bin/cli.py", line 925, in scheduler
> job.run()
> File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 196, in run
> session.commit()
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 927, in commit
> self.transaction.commit()
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 465, in commit
> self._assert_active(prepared_ok=True)
> File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 276, in _assert_active
> % self._rollback_exception
> sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.OperationalError) no such table: log [SQL: 'INSERT INTO log (dttm, dag_id, task_id, event, execution_date, owner, extra) VALUES (?, ?, ?, ?, ?, ?, ?)'] [parameters: ('2018-11-20 12:02:40.969353', None, None, 'cli_scheduler', None, 'airflow', '
> {"host_name": "airflow-airflow-scheduler-5b5f8b9549-89dmn", "full_command": "[\'/usr/local/bin/airflow\', \'scheduler\', \'-n\', \'-1\', \'-p\']"}
> ')]
> {color:#FF0000}*The pod status is like below*{color}
> [root@kubernetes-cpal-master-0 kube-airflow]# *kubectl get pod*
> NAME READY STATUS RESTARTS AGE
> airflow-airflow-flower-6668559cf7-ll5bn 1/1 Running 0 19m
> ~{color:#FF0000}*airflow-airflow-scheduler-5b5f8b9549-89dmn 0/1 CrashLoopBackOff 8 19m*{color}~
> airflow-airflow-web-89d8fb554-9ztss 1/1 Running 0 19m
> airflow-airflow-web-89d8fb554-f4mbm 1/1 Running 0 19m
> airflow-airflow-worker-0 1/1 Running 0 19m
> airflow-postgresql-9df799579-swg8q 1/1 Running 0 19m
> airflow-redis-7d75b85f7-26lsx 1/1 Running 0 19m
> centos 0/1 CrashLoopBackOff 1278 29d
> logger-deepinsights-fluentd-759ffcfc5d-fjmml 1/1 Running 8 33d
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)