You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/12/18 18:59:42 UTC
[GitHub] [airflow] MatrixManAtYrService opened a new issue #20400: `@task` decorated function passed to `python_callable` is treated as Truthy even if it returns False
MatrixManAtYrService opened a new issue #20400:
URL: https://github.com/apache/airflow/issues/20400
### Apache Airflow version
2.2.2 (latest released)
### What happened
I ran this dag:
```python3
@task
def run_downstream():
return False
@dag(
start_date=datetime(year=1970, month=1, day=1),
schedule_interval=None,
doc_md=dedent(
"""
If the user tries to treat a @task decorated function as a
python_callable, what happens?
"""
),
)
def should_skip():
ShortCircuitOperator(
task_id="decide", python_callable=run_downstream
) >> DummyOperator(task_id="skip_me")
the_dag = should_skip()
```
### What you expected to happen
At first, I expected the `ShortCircuitOperator` to see that `run_downstream` returns `False` and respond by setting `skip_me` to skipped. I was surprised when `skip_me` succeeded instead.
The task logs for `decide` show a warning:
```
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} INFO - Running <TaskInstance: should_skip.should_succeed manual__2021-12-18T18:25:37.029886+00:00 [running]> on host 7023b9855ace
[2021-12-18, 18:25:38 UTC] {taskinstance.py:1427} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=should_skip
AIRFLOW_CTX_TASK_ID=should_succeed
AIRFLOW_CTX_EXECUTION_DATE=2021-12-18T18:25:37.029886+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2021-12-18T18:25:37.029886+00:00
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - --- Logging error ---
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - Traceback (most recent call last):
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
msg = self.format(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
return fmt.format(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
record.message = record.getMessage()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
msg = msg % self.args
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/xcom_arg.py", line 85, in __str__
f"dag_id='{self.operator.dag.dag_id}'",
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/baseoperator.py", line 822, in dag
raise AirflowException(f'Operator {self} has not been assigned to a DAG yet')
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - airflow.exceptions.AirflowException: Operator <Task(_PythonDecoratedOperator): skip_downstream> has not been assigned to a DAG yet
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - Call stack:
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/bin/airflow", line 8, in <module>
sys.exit(main())
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/__main__.py", line 48, in main
args.func(args)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/scheduler_command.py", line 75, in scheduler
_run_scheduler_job(args=args)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/scheduler_command.py", line 46, in _run_scheduler_job
job.run()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/jobs/base_job.py", line 245, in run
self._execute()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line 620, in _execute
self.executor.start()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 356, in start
self.impl.start()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 298, in start
worker.start()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/popen_fork.py", line 71, in _launch
code = process_obj._bootstrap(parent_sentinel=child_r)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 67, in run
return super().run()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 190, in do_work
self.execute_work(key=key, command=command)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 84, in execute_work
state = self._execute_work_in_fork(command)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 124, in _execute_work_in_fork
args.func(args)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 292, in task_run
_run_task_by_selected_method(args, dag, ti)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 105, in _run_task_by_selected_method
_run_task_by_local_task_job(args, ti)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 163, in _run_task_by_local_task_job
run_job.run()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/jobs/base_job.py", line 245, in run
self._execute()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/jobs/local_task_job.py", line 103, in _execute
self.task_runner.start()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 41, in start
self.process = self._start_by_fork()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork
args.func(args, dag=self.dag)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 292, in task_run
_run_task_by_selected_method(args, dag, ti)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method
_run_raw_task(args, ti)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task
ti._run_raw_task(
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_task
self._execute_task_with_callbacks(context)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
result = execute_callable(context=context)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 212, in execute
condition = super().execute(context)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 152, in execute
self.log.info("Done. Returned value was: %s", return_value)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1446, in info
self._log(INFO, msg, args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1589, in _log
self.handle(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1599, in handle
self.callHandlers(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1661, in callHandlers
hdlr.handle(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 952, in handle
self.emit(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/log/file_task_handler.py", line 66, in emit
self.handler.emit(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - Message: 'Done. Returned value was: %s'
Arguments: (<airflow.models.xcom_arg.XComArg object at 0x7f079bc29100>,)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - --- Logging error ---
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - Traceback (most recent call last):
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
msg = self.format(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
return fmt.format(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
record.message = record.getMessage()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
msg = msg % self.args
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/xcom_arg.py", line 85, in __str__
f"dag_id='{self.operator.dag.dag_id}'",
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/baseoperator.py", line 822, in dag
raise AirflowException(f'Operator {self} has not been assigned to a DAG yet')
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - airflow.exceptions.AirflowException: Operator <Task(_PythonDecoratedOperator): skip_downstream> has not been assigned to a DAG yet
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - Call stack:
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/bin/airflow", line 8, in <module>
sys.exit(main())
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/__main__.py", line 48, in main
args.func(args)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/scheduler_command.py", line 75, in scheduler
_run_scheduler_job(args=args)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/scheduler_command.py", line 46, in _run_scheduler_job
job.run()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/jobs/base_job.py", line 245, in run
self._execute()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line 620, in _execute
self.executor.start()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 356, in start
self.impl.start()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 298, in start
worker.start()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/popen_fork.py", line 71, in _launch
code = process_obj._bootstrap(parent_sentinel=child_r)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 67, in run
return super().run()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 190, in do_work
self.execute_work(key=key, command=command)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 84, in execute_work
state = self._execute_work_in_fork(command)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/executors/local_executor.py", line 124, in _execute_work_in_fork
args.func(args)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 292, in task_run
_run_task_by_selected_method(args, dag, ti)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 105, in _run_task_by_selected_method
_run_task_by_local_task_job(args, ti)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 163, in _run_task_by_local_task_job
run_job.run()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/jobs/base_job.py", line 245, in run
self._execute()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/jobs/local_task_job.py", line 103, in _execute
self.task_runner.start()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 41, in start
self.process = self._start_by_fork()
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/task/task_runner/standard_task_runner.py", line 85, in _start_by_fork
args.func(args, dag=self.dag)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper
return f(*args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 292, in task_run
_run_task_by_selected_method(args, dag, ti)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 107, in _run_task_by_selected_method
_run_raw_task(args, ti)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/task_command.py", line 180, in _run_raw_task
ti._run_raw_task(
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1332, in _run_raw_task
self._execute_task_with_callbacks(context)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1458, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
result = execute_callable(context=context)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 213, in execute
self.log.info("Condition result is %s", condition)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1446, in info
self._log(INFO, msg, args, **kwargs)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1589, in _log
self.handle(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1599, in handle
self.callHandlers(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 1661, in callHandlers
hdlr.handle(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/logging/__init__.py", line 952, in handle
self.emit(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - File "/usr/local/lib/python3.9/site-packages/airflow/utils/log/file_task_handler.py", line 66, in emit
self.handler.emit(record)
[2021-12-18, 18:25:38 UTC] {logging_mixin.py:109} WARNING - Message: 'Condition result is %s'
Arguments: (<airflow.models.xcom_arg.XComArg object at 0x7f079bc29100>,)
[2021-12-18, 18:25:38 UTC] {python.py:216} INFO - Proceeding with downstream tasks...
[2021-12-18, 18:25:38 UTC] {taskinstance.py:1270} INFO - Marking task as SUCCESS. dag_id=should_skip, task_id=should_succeed, execution_date=20211218T182537, start_date=20211218T182538, end_date=20211218T182538
```
Then I saw my error: `run_downstream` isn't just a python_callable, it's `@task` decorated too. `ShortCircuitOperator` must be calling something other than the underlying function.
### How to reproduce
Run the dag shown above
### Operating System
docker/debian
### Versions of Apache Airflow Providers
n/a
### Deployment
Astronomer
### Deployment details
Dockerfile:
```
FROM quay.io/astronomer/ap-airflow-dev:2.2.2
```
### Anything else
This should probably do one of these:
- fail at parse time, and in a way that points out the user's error
- recognize that the `python_callable` happens to also be `@task` decorated and call the decorated function instead of the decorating function (and also emit a warning)
... or maybe something else entirely. I'm not sure. I do feel fairly confident that it should *not* do what it does, because it's hard to know what you've done wrong.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #20400: `@task` decorated function passed to `python_callable` is treated as Truthy even if it returns False
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #20400:
URL: https://github.com/apache/airflow/issues/20400
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20400: `@task` decorated function passed to `python_callable` is treated as Truthy even if it returns False
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20400:
URL: https://github.com/apache/airflow/issues/20400#issuecomment-1000407089
You should not really use @task -decorated function as callable. The decorator really returns a PythonOperator not the original function. If you need to use a logic that should be used in both callable and a decorated task, you should make a a non-decorated function and call it from within the decorated one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org