You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/27 21:03:13 UTC
[GitHub] [airflow] aliavni commented on issue #13322: Backfill crashes with "KeyError: TaskInstanceKey" when task has retries
aliavni commented on issue #13322:
URL: https://github.com/apache/airflow/issues/13322#issuecomment-849939475
I am having the same issue here on `airflow 2.0.2`. In my case, the dag works fine when deployed to the airflow server, but fails when I run or debug it in my IDE
The exception happens in this file `airflow/jobs/backfill_job.py:221` at the `ti_status.running.pop(reduced_key)` line. Right before the exception:
the value of `reduced_key` is:
```python
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 7, 0, 0, tzinfo=Timezone('UTC')), try_number=2)
```
and this is what I have in `ti_status.running`:
```python
{
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 1, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-01 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 2, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-02 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 3, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-03 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 4, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-04 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 5, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-05 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 6, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-06 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 7, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-07 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 8, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-08 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 9, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-09 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 10, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-10 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 11, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-11 00:00:00+00:00 [failed]>,
TaskInstanceKey(dag_id='refactor', task_id='task-1', execution_date=datetime.datetime(2021, 5, 12, 0, 0, tzinfo=Timezone('UTC')), try_number=3): <TaskInstance: refactor.task-1 2021-05-12 00:00:00+00:00 [failed]>
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org