You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/06 09:21:06 UTC
[GitHub] [airflow] chace20 opened a new issue #16831: try-finally segment can't work in operator
chace20 opened a new issue #16831:
URL: https://github.com/apache/airflow/issues/16831
**Apache Airflow version**: 2.0.1
**Environment**:
- **Cloud provider or hardware configuration**: x86_64
- **OS** (e.g. from /etc/os-release): Ubuntu 20.04
- **Kernel** (e.g. `uname -a`): 4.19.128-microsoft-standard
- **Install tools**: None
- **Others**: None
**What happened**:
Hi, I want to catch exception and do something, then raise this exception and mark task failed. But the task is success unexpectly.
Below it's my demo code:
``` python
from airflow.sensors.base import BaseSensorOperator
class DemoOperator(BaseSensorOperator):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def execute(self, context):
try:
new_ips = []
for i in range(10):
new_ips.append(i)
if i > 5:
raise Exception('some error')
except Exception as e:
print(e)
raise
finally:
print(new_ips)
return new_ips
```
The task log is:
``` log
[2021-07-06 17:02:33,988] {taskinstance.py:1063} INFO - Executing <Task(DemoOperator): demo> on 2021-07-06T09:02:32.951416+00:00
[2021-07-06 17:02:33,991] {standard_task_runner.py:52} INFO - Started process 9782 to run task
[2021-07-06 17:02:33,994] {standard_task_runner.py:76} INFO - Running: ['airflow', 'tasks', 'run', 'tutorial', 'demo', '2021-07-06T09:02:32.951416+00:00', '--job-id', '1457', '--pool', 'default_pool', '--raw', '--subdir', 'DAGS_FOLDER/demo.py', '--cfg-path', '/tmp/tmpsb18jv7b', '--error-file', '/tmp/tmpnkksn9yv']
[2021-07-06 17:02:33,994] {standard_task_runner.py:77} INFO - Job 1457: Subtask demo
[2021-07-06 17:02:34,025] {logging_mixin.py:104} INFO - Running <TaskInstance: tutorial.demo 2021-07-06T09:02:32.951416+00:00 [running]> on host 127.0.1.1
[2021-07-06 17:02:34,053] {taskinstance.py:1257} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_EMAIL=airflow@example.com
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=tutorial
AIRFLOW_CTX_TASK_ID=demo
AIRFLOW_CTX_EXECUTION_DATE=2021-07-06T09:02:32.951416+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2021-07-06T09:02:32.951416+00:00
[2021-07-06 17:02:34,054] {logging_mixin.py:104} INFO - some error
[2021-07-06 17:02:34,054] {logging_mixin.py:104} INFO - [0, 1, 2, 3, 4, 5, 6]
[2021-07-06 17:02:34,075] {taskinstance.py:1166} INFO - Marking task as SUCCESS. dag_id=tutorial, task_id=demo, execution_date=20210706T090232, start_date=20210706T090233, end_date=20210706T090234
[2021-07-06 17:02:34,095] {taskinstance.py:1220} INFO - 0 downstream tasks scheduled from follow-on schedule check
[2021-07-06 17:02:34,124] {local_task_job.py:146} INFO - Task exited with return code 0
```
**What you expected to happen**:
I expect the log should print `[0, 1, 2, 3, 4, 5, 6]` then raise a Exception and task fail. I'm confused.
**How to reproduce it**:
You can import my `DemoOperator` and run it in a dag to reproduce this bug.
**Anything else we need to know**:
Every time it will occur.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #16831: try-finally segment can't work in operator
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #16831:
URL: https://github.com/apache/airflow/issues/16831
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #16831: try-finally segment can't work in operator
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #16831:
URL: https://github.com/apache/airflow/issues/16831#issuecomment-874732681
NEVER use return in finally https://stackoverflow.com/questions/11164144/weird-try-except-else-finally-behavior-with-return-statements. This is not that well-known behaviour of python but this is how it works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #16831: try-finally segment can't work in operator
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #16831:
URL: https://github.com/apache/airflow/issues/16831#issuecomment-874603136
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #16831: try-finally segment can't work in operator
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #16831:
URL: https://github.com/apache/airflow/issues/16831#issuecomment-874603136
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org