You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/11/16 07:40:13 UTC
[GitHub] [airflow] Greetlist opened a new issue #19612: Clear a deffered task_instance which still in deferred State Do Not clear it's next_method
Greetlist opened a new issue #19612:
URL: https://github.com/apache/airflow/issues/19612
### Apache Airflow version
2.2.1
### Operating System
Ubuntu 16.04
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other
### Deployment details
_No response_
### What happened
When I mark a defferd task to Failed-State then clear it or clear a defferd task which in Defferd-State, this task is marked success immediately.
I check the task instance table, this task's next_method is not null when I clear this task.This cause task's execute function is replaced by next_method function.
### What you expected to happen
When I clear a defferd task, I want to re-judge it's trigger condition, re-execute the execute function.
### How to reproduce
Sensor
``` python
class ExternalTaskFromOtherAirflowSensor(BaseSensorOperator):
def execute(self, context):
trigger = ExternalTaskFromOtherAirflowTrigger(args)
self.defer(trigger=trigger, method_name="execute_complete")
def execute_complete(self, context, event=None):
return
```
Trigger
``` python
class ExternalTaskFromOtherAirflowTrigger(BaseTrigger):
def _get_task_instance_state(self):
sql_query = "SELECT state FROM {}.task_instance WHERE dag_id = '{}' and run_id like '%{}%';".format(
self.old_airflow_db_name,
self.old_external_dag_id,
self.old_external_task_id,
self.old_external_execution_date_str
)
async def run(self):
while self._get_task_instance_state() == False:
await asyncio.sleep(5)
yield TriggerEvent('success')
```
1. Clear this task when it is in **defferd** State.
2. Mark this task as **failed** and clear this task.
Both operations above will cause this task is marked as **success** instead of re-execute the execute function.
### Anything else
Similar issue:
#18146
#19120
I try to add this code in **models/taskinstance.py clear_task_instances function**
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] malthe commented on issue #19612: Clearing a defered task instance in deferred state does not clear its next_method
Posted by GitBox <gi...@apache.org>.
malthe commented on issue #19612:
URL: https://github.com/apache/airflow/issues/19612#issuecomment-1028065284
We can close this now then right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #19612: Clearing a defered task instance in deferred state does not clear its next_method
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #19612:
URL: https://github.com/apache/airflow/issues/19612#issuecomment-971359593
Your added code makes sense. Feel free to start a pull request for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #19612: Clear a deffered task_instance which still in deferred State Do Not clear it's next_method
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #19612:
URL: https://github.com/apache/airflow/issues/19612#issuecomment-969964439
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org