You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org> on 2019/05/17 20:03:04 UTC
[jira] [Updated] (AIRFLOW-1347) sqlalchemy.orm.exc.FlushError:
Can't update table task_instance using NULL for primary key
[ https://issues.apache.org/jira/browse/AIRFLOW-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aizhamal Nurmamat kyzy updated AIRFLOW-1347:
--------------------------------------------
Component/s: (was: db)
(was: DagRun)
database
> sqlalchemy.orm.exc.FlushError: Can't update table task_instance using NULL for primary key
> ------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-1347
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1347
> Project: Apache Airflow
> Issue Type: Bug
> Components: database
> Affects Versions: 1.8.0
> Environment: ubuntu 12.04
> mysql 5.7.18
> Reporter: Antoine Grouazel
> Priority: Major
> Attachments: bug_airflow.txt
>
>
> *Description of the issue*: the scheduler cannot update the mysql db via SqlAlchemy and the tasks are queued and not rescheduled. It affects all our DAGs but not all the tasks nor all the time. Manually clear a task having such issue can fix the problem for the given task instance.
> There is a kind of random behavior...
> Upgrading mysql 5.7.17 to 5.7.18 does not fix the issue. (x)
> Issue is neitheir fixed in the lastest version 1.8.2rc1 . (x)
> Don't know if there is a link with https://issues.apache.org/jira/browse/AIRFLOW-977. (?)
> {quote}
> {{[2017-06-24 00:31:10,272] {models.py:167} INFO - Filling up the DagBag from /home3/homedir7/perso/satwave/airflow/dags/sentinel1_copernicus_download.py
> [2017-06-24 00:31:10,427] {base_task_runner.py:112} INFO - Running: ['bash', '-c', u'airflow run Sentinel1_copernicus_download latest_only 2017-06-23T00:31:00 --job_id 293 --raw -sd DAGS_FOLDER/sentinel1_copernicus_download.py']
> [2017-06-24 00:31:11,449] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:11,449] {__init__.py:57} INFO - Using executor CeleryExecutor
> [2017-06-24 00:31:11,668] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:11,668] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
> [2017-06-24 00:31:11,724] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:11,723] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
> [2017-06-24 00:31:12,067] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:12,066] {models.py:167} INFO - Filling up the DagBag from /home3/homedir7/perso/satwave/airflow/dags/sentinel1_copernicus_download.py
> [2017-06-24 00:31:12,184] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:12,184] {models.py:1126} INFO - Dependencies all met for <TaskInstance: Sentinel1_copernicus_download.latest_only 2017-06-23 00:31:00 [queued]>
> [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:12,188] {models.py:1126} INFO - Dependencies all met for <TaskInstance: Sentinel1_copernicus_download.latest_only 2017-06-23 00:31:00 [queued]>
> [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: [2017-06-24 00:31:12,188] {models.py:1318} INFO -
> [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: --------------------------------------------------------------------------------
> [2017-06-24 00:31:12,188] {base_task_runner.py:95} INFO - Subtask: Starting attempt 1 of 3
> [2017-06-24 00:31:12,189] {base_task_runner.py:95} INFO - Subtask: --------------------------------------------------------------------------------
> [2017-06-24 00:31:12,189] {base_task_runner.py:95} INFO - Subtask:
> [2017-06-24 00:31:12,228] {base_task_runner.py:95} INFO - Subtask: Traceback (most recent call last):
> [2017-06-24 00:31:12,228] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/bin/airflow", line 28, in <module>
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: args.func(args)
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 422, in run
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: pool=args.pool,
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/utils/db.py", line 53, in wrapper
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: result = func(*args, **kwargs)
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1328, in run
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: session.commit()
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 801, in commit
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: self.transaction.commit()
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 392, in commit
> [2017-06-24 00:31:12,229] {base_task_runner.py:95} INFO - Subtask: self._prepare_impl()
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 372, in _prepare_impl
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: self.session.flush()
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2019, in flush
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: self._flush(objects)
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2137, in _flush
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: transaction.rollback(_capture_exception=True)
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: compat.reraise(exc_type, exc_value, exc_tb)
> [2017-06-24 00:31:12,230] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2101, in _flush
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: flush_context.execute()
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 373, in execute
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: rec.execute(self)
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 532, in execute
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: uow
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 170, in save_obj
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: mapper, table, update)
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 634, in _emit_update_statements
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: lambda rec: (
> [2017-06-24 00:31:12,231] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 531, in _collect_update_commands
> [2017-06-24 00:31:12,232] {base_task_runner.py:95} INFO - Subtask: "key value on column %s" % (table, col))
> [2017-06-24 00:31:12,232] {base_task_runner.py:95} INFO - Subtask: sqlalchemy.orm.exc.FlushError: Can't update table task_instance using NULL for primary key value on column task_instance.execution_date
> [2017-06-24 00:31:12,311] {jobs.py:2083} INFO - Task exited with return code 1}}{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)