You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/15 22:27:11 UTC
[GitHub] [airflow] anilkulkarni87 opened a new issue #14252: Unable to clear Failed task with retries
anilkulkarni87 opened a new issue #14252:
URL: https://github.com/apache/airflow/issues/14252
**Apache Airflow version**: 2.0.1
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): NA
**Environment**: Windows WSL2 (Ubuntu) Local
- **Cloud provider or hardware configuration**:
- **OS** (e.g. from /etc/os-release): Ubuntu 18.04
- **Kernel** (e.g. `uname -a`): Linux d255bce4dcd5 5.4.72-microsoft-standard-WSL2
- **Install tools**: Docker -compose
- **Others**:
**What happened**:
I have a dag with tasks:
Task1 - Get Date
Task 2 - Get data from Api call (Have set retires to 3)
Task 3 - Load Data
Task 2 had failed after three attempts. I am unable to clear the task Instance and get the below error in UI.
[Dag Code](https://github.com/anilkulkarni87/airflow-docker/blob/master/dags/covidNyDaily.py)
```
Python version: 3.8.7
Airflow version: 2.0.1rc2
Node: d255bce4dcd5
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/airflow/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/auth.py", line 34, in decorated
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/decorators.py", line 60, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/views.py", line 1547, in clear
return self._clear_dag_tis(
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/views.py", line 1475, in _clear_dag_tis
count = dag.clear(
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/session.py", line 65, in wrapper
return func(*args, session=session, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/dag.py", line 1324, in clear
clear_task_instances(
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 160, in clear_task_instances
ti.max_tries = ti.try_number + task_retries - 1
TypeError: unsupported operand type(s) for +: 'int' and 'str'
```
**What you expected to happen**:
I expected to clear the Task Instance so that the task could be scheduled again.
**How to reproduce it**:
1) Clone the repo link shared above
2) Follow instructions to setup cluster.
3) Change code to enforce error in Task 2
4) Execute and try to clear task instance after three attempts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #14252: Unable to clear Failed task with retries
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #14252:
URL: https://github.com/apache/airflow/issues/14252
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vshabarish edited a comment on issue #14252: Unable to clear Failed task with retries
Posted by GitBox <gi...@apache.org>.
vshabarish edited a comment on issue #14252:
URL: https://github.com/apache/airflow/issues/14252#issuecomment-819069341
we also have the same issues on Airflow 1.10.10, Python 3.6. This is happening only on few dags the clear function is not working anymore. Can someone help us try to fix the issues? And it is super important for our Production environments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #14252: Unable to clear Failed task with retries
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #14252:
URL: https://github.com/apache/airflow/issues/14252#issuecomment-819179008
Isn’t the `retries` value supposed to be an int? The repro above has
```python
@dag.task(default_args={'retries': '2', 'retry_delay': timedelta(minutes=30)})
```
Which is the cause to the exception, if I’m not mistaken.
That said, Airflow should probably be more resilient against user issue like this. Probably set the value to default with a warning?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vshabarish commented on issue #14252: Unable to clear Failed task with retries
Posted by GitBox <gi...@apache.org>.
vshabarish commented on issue #14252:
URL: https://github.com/apache/airflow/issues/14252#issuecomment-819716255
> Isn’t the `retries` value supposed to be an int? The repro above has
>
> ```python
> @dag.task(default_args={'retries': '2', 'retry_delay': timedelta(minutes=30)})
> ```
>
> Which is the cause to the exception, if I’m not mistaken.
>
> That said, Airflow should probably be more resilient against user issue like this. Probably set the value to default with a warning?
Excellent, it is working for us by changing the str to int at retries. Thanks much for the help !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hsnprsd commented on issue #14252: Unable to clear Failed task with retries
Posted by GitBox <gi...@apache.org>.
hsnprsd commented on issue #14252:
URL: https://github.com/apache/airflow/issues/14252#issuecomment-854628594
@uranusjr And if the retries is an string i think we can try to parse it as integer if we can.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] saimon46 commented on issue #14252: Unable to clear Failed task with retries
Posted by GitBox <gi...@apache.org>.
saimon46 commented on issue #14252:
URL: https://github.com/apache/airflow/issues/14252#issuecomment-795292533
I have the same problem with Airflow 2.0.1. I made an update from version 1.10.14 and the clear function is not working anymore. Can we try to fix this issue? I think it's a quite important function.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vshabarish commented on issue #14252: Unable to clear Failed task with retries
Posted by GitBox <gi...@apache.org>.
vshabarish commented on issue #14252:
URL: https://github.com/apache/airflow/issues/14252#issuecomment-819069341
we also have the same issues on Airflow 1.10.10, Python 3.6. This is happening only on few dags the clear function is not working anymore. Can we try to fix the issues? And it is super important for our Production environments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org