You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/06/04 23:49:36 UTC
[GitHub] [airflow] dimon222 opened a new issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
dimon222 opened a new issue #9148:
URL: https://github.com/apache/airflow/issues/9148
**Apache Airflow version**:
1.10.10
**Database version**:
MariaDB
**What happened**:
When task fails, next task retry raises this exception immediately:
<details><summary>INSERT INTO rendered_task_instance_fields (dag_id, task_id, execution_date, rendered_fields) VALUES (%s, %s, %s, %s)
('test_dag', 'test_task', datetime.datetime(2020, 6, 4, 22, 20, 21, 139775), '')
_mysql_exceptions.IntegrityError: (1062, "Duplicate entry 'test_dag-test_task-2020-' for key 'PRIMARY')</summary></details>
For some reason the value for execution date is calculated exactly same as previous retry. I might assume its not supposed to happen.
**What you expected to happen**:
Task retries correctly.
**How to reproduce it**:
Crash task that has retries > 1, wait for retry.
Alternatively, crash task, then clear this task's state in dag view to cause manual retry.
**Anything else we need to know**:
It happens always on retry. Not happening on 1.10.9 (very recent regression)
I'm able to retry correctly by removing related task records in MariaDB in rendered_task_instance_fields table.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #9148:
URL: https://github.com/apache/airflow/issues/9148
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-641337212
I will fix this for 1.10.11 soon and will try to post a workaround too in 1-2 days
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-639374661
Yes, will fix it for 1.10.11
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-639176799
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-644775583
Looks like this issues only occurs when using MySQL backend
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-639242210
@kaxil @ashb @anitakar -> this one seems to be serialization related and seems to be fairly important to fix for 1.10.11
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-639370078
@dimon222 Disabling `store_serialized_dags` in your config should also work around this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-639596214
@dimon222 Can you try setting `max_num_rendered_ti_fields_per_task` to 1 and see it that helps for now ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-645019460
Current workarounds for MySQL Users:
1) Disable DAG Serialization
2) Run the following query on your Metadata DB:
```sql
alter table rendered_task_instance_fields modify execution_date timestamp(6) not null;
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chrismclennon commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
chrismclennon commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-639998662
I'm experiencing the same issue. @kaxil I set `max_num_rendered_ti_fields_per_task=1` and am still seeing the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimon222 commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
dimon222 commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-641335335
> @dimon222 Can you try setting `max_num_rendered_ti_fields_per_task` under `[core]` section to 1 and see it that helps for now ?
Not helping, receiving same error. I'm afraid this setting is ignored.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil edited a comment on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-639596214
@dimon222 Can you try setting `max_num_rendered_ti_fields_per_task` under `[core]` section to 1 and see it that helps for now ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #9148: Retries are causing constraint violation (Airflow 1.10.10)
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #9148:
URL: https://github.com/apache/airflow/issues/9148#issuecomment-639369608
Yes, agreed on both counts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org