You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Laura Lorenz (JIRA)" <ji...@apache.org> on 2017/01/11 21:49:16 UTC
[jira] [Created] (AIRFLOW-748) Cannot upgradedb from airflow 1.7.0
to 1.8.0rca4
Laura Lorenz created AIRFLOW-748:
------------------------------------
Summary: Cannot upgradedb from airflow 1.7.0 to 1.8.0rca4
Key: AIRFLOW-748
URL: https://issues.apache.org/jira/browse/AIRFLOW-748
Project: Apache Airflow
Issue Type: Bug
Affects Versions: Airflow 1.8
Environment: Metadata database is mysql: mysqld Ver 5.5.47-0ubuntu0.14.04.1 for debian-linux-gnu on x86_64 ((Ubuntu))
Airflow version is 1.8.0rca4 from http://people.apache.org/~bolke/airflow-1.8.0a4.tar.gz
Reporter: Laura Lorenz
Hello. When trying out the new 1.8 alpha 4 I was unable to upgrade my database.
Several parts of the UI blow up without the upgrade, as is probably expected if the models changed significantly.
When I try to run an upgrade, I get the following traceback:
{code}
root@6137acea3ae0:/datadive# airflow upgradedb
[2017-01-11 21:43:20,531] {__init__.py:57} INFO - Using executor CeleryExecutor
[2017-01-11 21:43:20,604] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2017-01-11 21:43:20,625] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2017-01-11 21:43:20,754] {plugins_manager.py:75} INFO - Importing plugin module /datadive/plugins/neo4j.py
[2017-01-11 21:43:20,771] {plugins_manager.py:75} INFO - Importing plugin module /datadive/plugins/__init__.py
[2017-01-11 21:43:20,771] {plugins_manager.py:75} INFO - Importing plugin module /datadive/plugins/docs_view.py
[2017-01-11 21:43:20,772] {plugins_manager.py:75} INFO - Importing plugin module /datadive/plugins/mysqldive.py
[2017-01-11 21:43:20,809] {plugins_manager.py:95} INFO - Creating module airflow.operators.neo4j_plugin
[2017-01-11 21:43:20,809] {plugins_manager.py:95} INFO - Creating module airflow.hooks.neo4j_plugin
[2017-01-11 21:43:20,809] {plugins_manager.py:95} INFO - Creating module airflow.executors.neo4j_plugin
[2017-01-11 21:43:20,809] {plugins_manager.py:95} INFO - Creating module airflow.macros.neo4j_plugin
[2017-01-11 21:43:20,809] {plugins_manager.py:95} INFO - Creating module airflow.operators.docs
[2017-01-11 21:43:20,809] {plugins_manager.py:95} INFO - Creating module airflow.hooks.docs
[2017-01-11 21:43:20,810] {plugins_manager.py:95} INFO - Creating module airflow.executors.docs
[2017-01-11 21:43:20,810] {plugins_manager.py:95} INFO - Creating module airflow.macros.docs
[2017-01-11 21:43:20,810] {plugins_manager.py:95} INFO - Creating module airflow.operators.mysqldive_plugin
[2017-01-11 21:43:20,810] {plugins_manager.py:95} INFO - Creating module airflow.hooks.mysqldive_plugin
[2017-01-11 21:43:20,810] {plugins_manager.py:95} INFO - Creating module airflow.executors.mysqldive_plugin
[2017-01-11 21:43:20,810] {plugins_manager.py:95} INFO - Creating module airflow.macros.mysqldive_plugin
DB: mysql://airflow:***@mysql/airflow
[2017-01-11 21:43:20,862] {db.py:284} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade f2ca10b85618 -> 4addfa1236f1, Add fractional seconds to mysql tables
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/bin/airflow", line 28, in <module>
args.func(args)
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/bin/cli.py", line 883, in upgradedb
db_utils.upgradedb()
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/utils/db.py", line 292, in upgradedb
command.upgrade(config, 'heads')
File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 174, in upgrade
script.run_env()
File "/usr/local/lib/python2.7/dist-packages/alembic/script/base.py", line 407, in run_env
util.load_python_file(self.dir, 'env.py')
File "/usr/local/lib/python2.7/dist-packages/alembic/util/pyfiles.py", line 93, in load_python_file
module = load_module_py(module_id, path)
File "/usr/local/lib/python2.7/dist-packages/alembic/util/compat.py", line 79, in load_module_py
mod = imp.load_source(module_id, path, fp)
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/migrations/env.py", line 88, in <module>
run_migrations_online()
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/migrations/env.py", line 83, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/usr/local/lib/python2.7/dist-packages/alembic/runtime/environment.py", line 797, in run_migrations
self.get_context().run_migrations(**kw)
File "/usr/local/lib/python2.7/dist-packages/alembic/runtime/migration.py", line 312, in run_migrations
step.migration_fn(**kw)
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/migrations/versions/4addfa1236f1_add_fractional_seconds_to_mysql_tables.py", line 36, in upgrade
op.alter_column(table_name='dag', column_name='last_scheduler_run', type_=mysql.DATETIME(fsp=6))
File "<string>", line 8, in alter_column
File "<string>", line 3, in alter_column
File "/usr/local/lib/python2.7/dist-packages/alembic/operations/ops.py", line 1420, in alter_column
return operations.invoke(alt)
File "/usr/local/lib/python2.7/dist-packages/alembic/operations/base.py", line 318, in invoke
return fn(self, operation)
File "/usr/local/lib/python2.7/dist-packages/alembic/operations/toimpl.py", line 53, in alter_column
**operation.kw
File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/mysql.py", line 67, in alter_column
else existing_autoincrement
File "/usr/local/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 118, in _exec
return conn.execute(construct, *multiparams, **params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
compiled
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 226, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorvalue
sqlalchemy.exc.ProgrammingError: (_mysql_exceptions.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) NULL' at line 1") [SQL: u'ALTER TABLE dag MODIFY last_scheduler_run DATETIME(6) NULL']
{code}
I assume the upgrade failure is the cause of this error I see from the webserver logs when I try to navigate to a DAG graph or the task instance list, for example:
{code}
[2017-01-11 21:43:53,032] {__init__.py:57} INFO - Using executor CeleryExecutor
[2017-01-11 21:43:53,104] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2017-01-11 21:43:53,125] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2017-01-11 21:43:53,252] {plugins_manager.py:75} INFO - Importing plugin module /datadive/plugins/neo4j.py
[2017-01-11 21:43:53,268] {plugins_manager.py:75} INFO - Importing plugin module /datadive/plugins/__init__.py
[2017-01-11 21:43:53,268] {plugins_manager.py:75} INFO - Importing plugin module /datadive/plugins/docs_view.py
[2017-01-11 21:43:53,269] {plugins_manager.py:75} INFO - Importing plugin module /datadive/plugins/mysqldive.py
[2017-01-11 21:43:53,306] {plugins_manager.py:95} INFO - Creating module airflow.operators.neo4j_plugin
[2017-01-11 21:43:53,306] {plugins_manager.py:95} INFO - Creating module airflow.hooks.neo4j_plugin
[2017-01-11 21:43:53,306] {plugins_manager.py:95} INFO - Creating module airflow.executors.neo4j_plugin
[2017-01-11 21:43:53,306] {plugins_manager.py:95} INFO - Creating module airflow.macros.neo4j_plugin
[2017-01-11 21:43:53,307] {plugins_manager.py:95} INFO - Creating module airflow.operators.docs
[2017-01-11 21:43:53,307] {plugins_manager.py:95} INFO - Creating module airflow.hooks.docs
[2017-01-11 21:43:53,307] {plugins_manager.py:95} INFO - Creating module airflow.executors.docs
[2017-01-11 21:43:53,307] {plugins_manager.py:95} INFO - Creating module airflow.macros.docs
[2017-01-11 21:43:53,307] {plugins_manager.py:95} INFO - Creating module airflow.operators.mysqldive_plugin
[2017-01-11 21:43:53,307] {plugins_manager.py:95} INFO - Creating module airflow.hooks.mysqldive_plugin
[2017-01-11 21:43:53,307] {plugins_manager.py:95} INFO - Creating module airflow.executors.mysqldive_plugin
[2017-01-11 21:43:53,307] {plugins_manager.py:95} INFO - Creating module airflow.macros.mysqldive_plugin
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
[2017-01-11 21:43:53,411] {jobs.py:1222} INFO - Starting the scheduler
[2017-01-11 21:43:53,411] {jobs.py:1238} INFO - Processing files using up to 2 processes at a time
[2017-01-11 21:43:53,411] {jobs.py:1240} INFO - Running execute loop for -1 seconds
[2017-01-11 21:43:53,411] {jobs.py:1242} INFO - Processing each file at most None times
[2017-01-11 21:43:53,412] {jobs.py:1244} INFO - Process each file at most once every 0 seconds
[2017-01-11 21:43:53,412] {jobs.py:1246} INFO - Checking for new files in /datadive/dags every 300 seconds
[2017-01-11 21:43:53,412] {jobs.py:1249} INFO - Searching for files in /datadive/dags
[2017-01-11 21:43:53,413] {jobs.py:1252} INFO - There are 15 files in /datadive/dags
[2017-01-11 21:43:53,413] {jobs.py:1314} INFO - Resetting state for orphaned tasks
[2017-01-11 21:43:53,415] {jobs.py:1323} INFO - Resetting timeseries_demographic 2016-10-03 00:00:00
[2017-01-11 21:43:53,417] {jobs.py:1271} INFO - Exited execute loop
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/bin/airflow", line 28, in <module>
args.func(args)
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/bin/cli.py", line 796, in scheduler
job.run()
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/jobs.py", line 202, in run
self._execute()
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/jobs.py", line 1269, in _execute
self._execute_helper(processor_manager)
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/jobs.py", line 1324, in _execute_helper
self._reset_state_for_orphaned_tasks(dr, session=session)
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/utils/db.py", line 54, in wrapper
result = func(*args, **kwargs)
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/jobs.py", line 1212, in _reset_state_for_orphaned_tasks
tis.extend(dag_run.get_task_instances(state=State.SCHEDULED, session=session))
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/utils/db.py", line 54, in wrapper
result = func(*args, **kwargs)
File "/datadive/airflow-1.8.0a4/airflow-1.8.0a4/airflow/models.py", line 3802, in get_task_instances
return tis.all()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2613, in all
return list(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2761, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2776, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 226, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorvalue
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1054, "Unknown column 'task_instance.pid' in 'field list'") [SQL: u'SELECT task_instance.task_id AS task_instance_task_id, task_instance.dag_id AS task_instance_dag_id, task_instance.execution_date AS task_instance_execution_date, task_instance.start_date AS task_instance_start_date, task_instance.end_date AS task_instance_end_date, task_instance.duration AS task_instance_duration, task_instance.state AS task_instance_state, task_instance.try_number AS task_instance_try_number, task_instance.hostname AS task_instance_hostname, task_instance.unixname AS task_instance_unixname, task_instance.job_id AS task_instance_job_id, task_instance.pool AS task_instance_pool, task_instance.queue AS task_instance_queue, task_instance.priority_weight AS task_instance_priority_weight, task_instance.operator AS task_instance_operator, task_instance.queued_dttm AS task_instance_queued_dttm, task_instance.pid AS task_instance_pid \nFROM task_instance \nWHERE task_instance.dag_id = %s AND task_instance.execution_date = %s AND task_instance.state = %s'] [parameters: ('timeseries_demographic', datetime.datetime(2016, 10, 3, 0, 0), u'scheduled')]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)