You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Zach Liu (Jira)" <ji...@apache.org> on 2019/09/09 21:14:00 UTC
[jira] [Commented] (AIRFLOW-1370) Scheduler is crashing because of
IntegrityError
[ https://issues.apache.org/jira/browse/AIRFLOW-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16926100#comment-16926100 ]
Zach Liu commented on AIRFLOW-1370:
-----------------------------------
Using airflow 1.10.5, i got a similar error when switching on a dag from the UI, it seems 2 postgres sessions are trying to create a table at the same time
{code:python}
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(task_id_sequence, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT: CREATE SEQUENCE task_id_sequence
scheduler_1_4c2247cc7f73 | [2019-09-09 20:52:18,139] {__init__.py:58} WARNING - Failed operation _get_task_meta_for. Retrying 2 more times.
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(task_id_sequence, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT:
postgres_1_b6eb2ed4d926 | CREATE TABLE celery_taskmeta (
postgres_1_b6eb2ed4d926 | id INTEGER NOT NULL,
postgres_1_b6eb2ed4d926 | task_id VARCHAR(155),
postgres_1_b6eb2ed4d926 | status VARCHAR(50),
postgres_1_b6eb2ed4d926 | result BYTEA,
postgres_1_b6eb2ed4d926 | date_done TIMESTAMP WITHOUT TIME ZONE,
postgres_1_b6eb2ed4d926 | traceback TEXT,
postgres_1_b6eb2ed4d926 | PRIMARY KEY (id),
postgres_1_b6eb2ed4d926 | UNIQUE (task_id)
postgres_1_b6eb2ed4d926 | )
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT:
postgres_1_b6eb2ed4d926 | CREATE TABLE celery_taskmeta (
postgres_1_b6eb2ed4d926 | id INTEGER NOT NULL,
postgres_1_b6eb2ed4d926 | task_id VARCHAR(155),
postgres_1_b6eb2ed4d926 | status VARCHAR(50),
postgres_1_b6eb2ed4d926 | result BYTEA,
postgres_1_b6eb2ed4d926 | date_done TIMESTAMP WITHOUT TIME ZONE,
postgres_1_b6eb2ed4d926 | traceback TEXT,
postgres_1_b6eb2ed4d926 | PRIMARY KEY (id),
postgres_1_b6eb2ed4d926 | UNIQUE (task_id)
postgres_1_b6eb2ed4d926 | )
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT:
postgres_1_b6eb2ed4d926 | CREATE TABLE celery_taskmeta (
postgres_1_b6eb2ed4d926 | id INTEGER NOT NULL,
postgres_1_b6eb2ed4d926 | task_id VARCHAR(155),
postgres_1_b6eb2ed4d926 | status VARCHAR(50),
postgres_1_b6eb2ed4d926 | result BYTEA,
postgres_1_b6eb2ed4d926 | date_done TIMESTAMP WITHOUT TIME ZONE,
postgres_1_b6eb2ed4d926 | traceback TEXT,
postgres_1_b6eb2ed4d926 | PRIMARY KEY (id),
postgres_1_b6eb2ed4d926 | UNIQUE (task_id)
postgres_1_b6eb2ed4d926 | )
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT:
postgres_1_b6eb2ed4d926 | CREATE TABLE celery_taskmeta (
postgres_1_b6eb2ed4d926 | id INTEGER NOT NULL,
postgres_1_b6eb2ed4d926 | task_id VARCHAR(155),
postgres_1_b6eb2ed4d926 | status VARCHAR(50),
postgres_1_b6eb2ed4d926 | result BYTEA,
postgres_1_b6eb2ed4d926 | date_done TIMESTAMP WITHOUT TIME ZONE,
postgres_1_b6eb2ed4d926 | traceback TEXT,
postgres_1_b6eb2ed4d926 | PRIMARY KEY (id),
postgres_1_b6eb2ed4d926 | UNIQUE (task_id)
postgres_1_b6eb2ed4d926 | )
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 |
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 815, in visit_table
scheduler_1_4c2247cc7f73 | self.traverse_single(column.default)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 859, in visit_sequence
scheduler_1_4c2247cc7f73 | self.connection.execute(CreateSequence(sequence))
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(task_id_sequence, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL: CREATE SEQUENCE task_id_sequence]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 | [2019-09-09 20:52:18,183] {__init__.py:58} WARNING - Failed operation _get_task_meta_for. Retrying 1 more times.
scheduler_1_4c2247cc7f73 | [2019-09-09 20:52:18,181] {__init__.py:58} WARNING - Failed operation _get_task_meta_for. Retrying 2 more times.
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT:
postgres_1_b6eb2ed4d926 | CREATE TABLE celery_tasksetmeta (
postgres_1_b6eb2ed4d926 | id INTEGER NOT NULL,
postgres_1_b6eb2ed4d926 | taskset_id VARCHAR(155),
postgres_1_b6eb2ed4d926 | result BYTEA,
postgres_1_b6eb2ed4d926 | date_done TIMESTAMP WITHOUT TIME ZONE,
postgres_1_b6eb2ed4d926 | PRIMARY KEY (id),
postgres_1_b6eb2ed4d926 | UNIQUE (taskset_id)
postgres_1_b6eb2ed4d926 | )
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT:
postgres_1_b6eb2ed4d926 | CREATE TABLE celery_tasksetmeta (
postgres_1_b6eb2ed4d926 | id INTEGER NOT NULL,
postgres_1_b6eb2ed4d926 | taskset_id VARCHAR(155),
postgres_1_b6eb2ed4d926 | result BYTEA,
postgres_1_b6eb2ed4d926 | date_done TIMESTAMP WITHOUT TIME ZONE,
postgres_1_b6eb2ed4d926 | PRIMARY KEY (id),
postgres_1_b6eb2ed4d926 | UNIQUE (taskset_id)
postgres_1_b6eb2ed4d926 | )
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT:
postgres_1_b6eb2ed4d926 | CREATE TABLE celery_tasksetmeta (
postgres_1_b6eb2ed4d926 | id INTEGER NOT NULL,
postgres_1_b6eb2ed4d926 | taskset_id VARCHAR(155),
postgres_1_b6eb2ed4d926 | result BYTEA,
postgres_1_b6eb2ed4d926 | date_done TIMESTAMP WITHOUT TIME ZONE,
postgres_1_b6eb2ed4d926 | PRIMARY KEY (id),
postgres_1_b6eb2ed4d926 | UNIQUE (taskset_id)
postgres_1_b6eb2ed4d926 | )
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 | ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
postgres_1_b6eb2ed4d926 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
postgres_1_b6eb2ed4d926 | STATEMENT:
postgres_1_b6eb2ed4d926 | CREATE TABLE celery_tasksetmeta (
postgres_1_b6eb2ed4d926 | id INTEGER NOT NULL,
postgres_1_b6eb2ed4d926 | taskset_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_taskmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | task_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | status VARCHAR(50),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | traceback TEXT,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (task_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
postgres_1_b6eb2ed4d926 | result BYTEA,
postgres_1_b6eb2ed4d926 | date_done TIMESTAMP WITHOUT TIME ZONE,
postgres_1_b6eb2ed4d926 | PRIMARY KEY (id),
postgres_1_b6eb2ed4d926 | UNIQUE (taskset_id)
postgres_1_b6eb2ed4d926 | )
postgres_1_b6eb2ed4d926 |
postgres_1_b6eb2ed4d926 |
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_taskmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | task_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | status VARCHAR(50),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | traceback TEXT,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (task_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 | [2019-09-09 20:52:18,185[] {__init__.py:2019-09-09 20:52:18,18658} WARNING] { - Failed operation _get_task_meta_for__init__.py:58} WARNING. Retrying 2 more times. - Failed operation
scheduler_1_4c2247cc7f73 | _get_task_meta_for. Retrying 2 more times.
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_taskmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | task_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | status VARCHAR(50),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | traceback TEXT,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (task_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_taskmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_taskmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | task_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | status VARCHAR(50),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | traceback TEXT,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (task_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 | [2019-09-09 20:52:18,236] {__init__.py:58} WARNING[2019-09-09 20:52:18,238] {__init__.py: - Failed operation _get_task_meta_for. Retrying 1 more times.
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_tasksetmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | taskset_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (taskset_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)58} WARNING - Failed operation _get_task_meta_for. Retrying 1 more times.
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_tasksetmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | taskset_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (taskset_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [2019-09-09 20:52:18,239] {__init__.py:58} WARNING - Failed operation _get_task_meta_for. Retrying 1 more times.
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_tasksetmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | taskset_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (taskset_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 | [2019-09-09 20:52:18,249] {__init__.py:58} WARNING - Failed operation _get_task_meta_for. Retrying 0 more times.
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_tasksetmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | taskset_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (taskset_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 | [2019-09-09 20:52:18,440] {celery_executor.py:264} ERROR - Error fetching Celery task state, ignoring it:IntegrityError('(psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"\nDETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.\n',)
scheduler_1_4c2247cc7f73 | Celery Task ID: ('test_ecs_op', 'gcs-to-s3-drds', datetime.datetime(2019, 9, 5, 0, 0, tzinfo=<TimezoneInfo [UTC, GMT, +00:00:00, STD]>), 1)
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | psycopg2.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | The above exception was the direct cause of the following exception:
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | Traceback (most recent call last):
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/airflow/executors/celery_executor.py", line 106, in fetch_celery_task_state
scheduler_1_4c2247cc7f73 | res = (celery_task[0], celery_task[1].state)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/result.py", line 473, in state
scheduler_1_4c2247cc7f73 | return self._get_task_meta()['status']
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/result.py", line 412, in _get_task_meta
scheduler_1_4c2247cc7f73 | return self._maybe_set_cache(self.backend.get_task_meta(self.id))
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/base.py", line 386, in get_task_meta
scheduler_1_4c2247cc7f73 | meta = self._get_task_meta_for(task_id)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 53, in _inner
scheduler_1_4c2247cc7f73 | return fun(*args, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 122, in _get_task_meta_for
scheduler_1_4c2247cc7f73 | session = self.ResultSession()
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/__init__.py", line 99, in ResultSession
scheduler_1_4c2247cc7f73 | **self.engine_options)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 59, in session_factory
scheduler_1_4c2247cc7f73 | self.prepare_models(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/celery/backends/database/session.py", line 54, in prepare_models
scheduler_1_4c2247cc7f73 | ResultModelBase.metadata.create_all(engine)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4294, in create_all
scheduler_1_4c2247cc7f73 | ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2046, in _run_visitor
scheduler_1_4c2247cc7f73 | conn._run_visitor(visitorcallable, element, **kwargs)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1615, in _run_visitor
scheduler_1_4c2247cc7f73 | visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
scheduler_1_4c2247cc7f73 | _is_metadata_operation=True,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 132, in traverse_single
scheduler_1_4c2247cc7f73 | return meth(obj, **kw)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
scheduler_1_4c2247cc7f73 | include_foreign_key_constraints,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
scheduler_1_4c2247cc7f73 | return meth(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
scheduler_1_4c2247cc7f73 | return connection._execute_ddl(self, multiparams, params)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
scheduler_1_4c2247cc7f73 | compiled,
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1253, in _execute_context
scheduler_1_4c2247cc7f73 | e, statement, parameters, cursor, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
scheduler_1_4c2247cc7f73 | util.raise_from_cause(sqlalchemy_exception, exc_info)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
scheduler_1_4c2247cc7f73 | reraise(type(exception), exception, tb=exc_tb, cause=cause)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
scheduler_1_4c2247cc7f73 | raise value.with_traceback(tb)
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1249, in _execute_context
scheduler_1_4c2247cc7f73 | cursor, statement, parameters, context
scheduler_1_4c2247cc7f73 | File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
scheduler_1_4c2247cc7f73 | cursor.execute(statement, parameters)
scheduler_1_4c2247cc7f73 | sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
scheduler_1_4c2247cc7f73 | DETAIL: Key (typname, typnamespace)=(celery_tasksetmeta, 2200) already exists.
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | [SQL:
scheduler_1_4c2247cc7f73 | CREATE TABLE celery_tasksetmeta (
scheduler_1_4c2247cc7f73 | id INTEGER NOT NULL,
scheduler_1_4c2247cc7f73 | taskset_id VARCHAR(155),
scheduler_1_4c2247cc7f73 | result BYTEA,
scheduler_1_4c2247cc7f73 | date_done TIMESTAMP WITHOUT TIME ZONE,
scheduler_1_4c2247cc7f73 | PRIMARY KEY (id),
scheduler_1_4c2247cc7f73 | UNIQUE (taskset_id)
scheduler_1_4c2247cc7f73 | )
scheduler_1_4c2247cc7f73 |
scheduler_1_4c2247cc7f73 | ]
scheduler_1_4c2247cc7f73 | (Background on this error at: http://sqlalche.me/e/gkpj)
scheduler_1_4c2247cc7f73 |
{code}
> Scheduler is crashing because of IntegrityError
> -----------------------------------------------
>
> Key: AIRFLOW-1370
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1370
> Project: Apache Airflow
> Issue Type: Bug
> Components: celery, scheduler
> Affects Versions: 1.8.0
> Reporter: Maneesh Sharma
> Priority: Major
>
> Scheduler is crashing with multiple task running on Celery Executor. It is throwing `{color:red}IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "task_instance_pkey"{color}`. Below is the complete stack trace of error --
> Process DagFileProcessor490-Process:
> Traceback (most recent call last):
> File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
> self.run()
> File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
> self._target(*self._args, **self._kwargs)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/jobs.py", line 348, in helper
> pickle_dags)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper
> result = func(*args, **kwargs)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/jobs.py", line 1587, in process_file
> self._process_dags(dagbag, dags, ti_keys_to_schedule)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/jobs.py", line 1176, in _process_dags
> self._process_task_instances(dag, tis_out)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/jobs.py", line 880, in _process_task_instances
> run.verify_integrity(session=session)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper
> result = func(*args, **kwargs)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 4117, in verify_integrity
> session.commit()
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 906, in commit
> self.transaction.commit()
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 461, in commit
> self._prepare_impl()
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 441, in _prepare_impl
> self.session.flush()
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2171, in flush
> self._flush(objects)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2291, in _flush
> transaction.rollback(_capture_exception=True)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
> compat.reraise(exc_type, exc_value, exc_tb)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2255, in _flush
> flush_context.execute()
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 389, in execute
> rec.execute(self)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 548, in execute
> uow
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
> mapper, table, insert)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 799, in _emit_insert_statements
> execute(statement, multiparams)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute
> return meth(self, multiparams, params)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
> compiled_sql, distilled_params
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
> context)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
> exc_info
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
> reraise(type(exception), exception, tb=exc_tb, cause=cause)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1159, in _execute_context
> context)
> File "/home/ubuntu/.local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 467, in do_executemany
> cursor.executemany(statement, parameters)
> IntegrityError: (psycopg2.IntegrityError) duplicate key value violates unique constraint "task_instance_pkey"
> DETAIL: Key (task_id, dag_id, execution_date)=(Hello_World_task-21, Hello_World_Tasks, 2017-07-04 06:59:40) already exists.
> [SQL: 'INSERT INTO task_instance (task_id, dag_id, execution_date, start_date, end_date, duration, state, try_number, hostname, unixname, job_id, pool, queue, priority_weight, operator, queued_dttm, pid) VALUES (%(task_id)s, %(dag_id)s, %(execution_date)s, %(start_date)s, %(end_date)s, %(duration)s, %(state)s, %(try_number)s, %(hostname)s, %(unixname)s, %(job_id)s, %(pool)s, %(queue)s, %(priority_weight)s, %(operator)s, %(queued_dttm)s, %(pid)s)'] [parameters: ({'task_id': 'Hello_World_task-21', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'}, {'task_id': 'Hello_World_task-20', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'}, {'task_id': 'Hello_World_task-23', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'}, {'task_id': 'Hello_World_task-22', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'}, {'task_id': 'Hello_World_task-25', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'}, {'task_id': 'Hello_World_task-24', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'}, {'task_id': 'Hello_World_task-27', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'}, {'task_id': 'Hello_World_task-26', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'} ... displaying 10 of 2002 total bound parameter sets ... {'task_id': 'Hello_World_task-1783', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'}, {'task_id': 'Hello_World_task-1782', 'unixname': 'ubuntu', 'job_id': None, 'end_date': None, 'pool': None, 'queued_dttm': None, 'execution_date': datetime.datetime(2017, 7, 4, 6, 59, 40), 'pid': None, 'try_number': 0, 'queue': 'default', 'duration': None, 'state': None, 'start_date': None, 'operator': None, 'priority_weight': 2, 'hostname': u'', 'dag_id': 'Hello_World_Tasks'})]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)