You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/13 10:32:37 UTC
[GitHub] [airflow] Kaash opened a new issue #11493: MSSQL backend broken when DAG serialization enabled
Kaash opened a new issue #11493:
URL: https://github.com/apache/airflow/issues/11493
Version is 1.10.12
Back-end is MSSQL 12
The following SQL query is broken:
result = session.query(cls.rendered_fields).filter(
cls.dag_id == ti.dag_id,
cls.task_id == ti.task_id,
cls.execution_date == ti.execution_date
).one_or_none()
This query fails as the rendered_task_instance_fields table in the database stores execution date as TIMESTAMP but other tables have updated execution date for MSSQL to DATETIME2
Here's the stack trace:
[2020-10-12 14:23:49,025] {taskinstance.py:1150} ERROR - (pyodbc.DataError) ('22018', '[22018] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Operand type clash: datetime2 is incompatible with timestamp (206) (SQLExecDirectW)')
[SQL: SELECT rendered_task_instance_fields.dag_id AS rendered_task_instance_fields_dag_id, rendered_task_instance_fields.task_id AS rendered_task_instance_fields_task_id, rendered_task_instance_fields.execution_date AS rendered_task_instance_fields_execution_date, rendered_task_instance_fields.rendered_fields AS rendered_task_instance_fields_rendered_fields
FROM rendered_task_instance_fields
WHERE rendered_task_instance_fields.dag_id = ? AND rendered_task_instance_fields.task_id = ? AND rendered_task_instance_fields.execution_date = ?]
[parameters: ('xxx-20200901', 'DATA_FEED_AVAILABLE_CHECK', <Pendulum [2020-10-09T00:00:00+00:00]>)]
(Background on this error at: http://sqlalche.me/e/13/9h9h)
Traceback (most recent call last):
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 596, in do_execute
cursor.execute(statement, parameters)
pyodbc.DataError: ('22018', '[22018] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Operand type clash: datetime2 is incompatible with timestamp (206) (SQLExecDirectW)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 967, in _run_raw_task
RTIF.write(RTIF(ti=self, render_templates=False), session=session)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/airflow/utils/db.py", line 70, in wrapper
return func(*args, **kwargs)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/airflow/models/renderedtifields.py", line 94, in write
session.merge(self)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2155, in merge
_resolve_conflict_map=_resolve_conflict_map,
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2228, in _merge
merged = self.query(mapper.class_).get(key[1])
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1018, in get
return self._get_impl(ident, loading.load_on_pk_identity)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 1135, in _get_impl
return db_load_fn(self, primary_key_identity)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 286, in load_on_pk_identity
return q.one()
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3463, in one
ret = self.one_or_none()
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3432, in one_or_none
ret = list(self)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3508, in __iter__
return self._execute_and_instances(context)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3533, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/local/scratch/khkaas/conda/envs/airflow-final/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
distilled_params,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb closed issue #11493: MSSQL backend broken when DAG serialization enabled
Posted by GitBox <gi...@apache.org>.
ashb closed issue #11493:
URL: https://github.com/apache/airflow/issues/11493
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Kaash commented on issue #11493: MSSQL backend broken when DAG serialization enabled
Posted by GitBox <gi...@apache.org>.
Kaash commented on issue #11493:
URL: https://github.com/apache/airflow/issues/11493#issuecomment-707914520
Have submitted this PR to resolve this issue: https://github.com/apache/airflow/pull/11512
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #11493: MSSQL backend broken when DAG serialization enabled
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #11493:
URL: https://github.com/apache/airflow/issues/11493#issuecomment-718831948
Fixed in 2.0.0a2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #11493: MSSQL backend broken when DAG serialization enabled
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #11493:
URL: https://github.com/apache/airflow/issues/11493#issuecomment-707649667
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org