You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by je...@apache.org on 2025/09/24 03:05:17 UTC
(airflow) branch main updated: Fix `example_bash_decorator` DAG (#56020)
This is an automated email from the ASF dual-hosted git repository.
jedcunningham pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new ab48b2b6ef5 Fix `example_bash_decorator` DAG (#56020)
ab48b2b6ef5 is described below
commit ab48b2b6ef5b6373ea286ecdee98a5961e49e6e4
Author: Kaxil Naik <ka...@apache.org>
AuthorDate: Wed Sep 24 04:05:10 2025 +0100
Fix `example_bash_decorator` DAG (#56020)
The ``example_bash_decorator`` was broken since it relied on the Airflow source code path.
Before:
```
{"timestamp":"2025-09-23T11:44:15.971920Z","level":"info","event":"Running command: ['/usr/bin/bash', '-c', 'stat']","logger":"airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook","filename":"subprocess.py","lineno":88}
{"timestamp":"2025-09-23T11:44:15.995649Z","level":"info","event":"Output:","logger":"airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook","filename":"subprocess.py","lineno":99}
{"timestamp":"2025-09-23T11:44:15.999568Z","level":"info","event":"stat: missing operand","logger":"airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook","filename":"subprocess.py","lineno":106}
{"timestamp":"2025-09-23T11:44:15.999669Z","level":"info","event":"Try 'stat --help' for more information.","logger":"airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook","filename":"subprocess.py","lineno":106}
{"timestamp":"2025-09-23T11:44:16.000278Z","level":"info","event":"Command exited with return code 1","logger":"airflow.task.hooks.airflow.providers.standard.hooks.subprocess.SubprocessHook","filename":"subprocess.py","lineno":110}
{"timestamp":"2025-09-23T11:44:16.000884Z","level":"error","event":"Task failed with exception","logger":"task","filename":"task_runner.py","lineno":972,"error_detail":[{"exc_type":"AirflowException","exc_value":"Bash command failed. The command returned a non-zero exit code 1.","exc_notes":[],"syntax_error":null,"is_cause":false,"frames":[{"filename":"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py","lineno":920,"name":"run"},{"filename":"/ [...]
{"timestamp":"2025-09-23T11:44:16.025800Z","level":"info","event":"Task instance in failure state","logger":"task.stdout"}
{"timestamp":"2025-09-23T11:44:16.025944Z","level":"info","event":"Task start","logger":"task.stdout"}
{"timestamp":"2025-09-23T11:44:16.026033Z","level":"info","event":"Task:<Task(_BashDecoratedOperator): get_file_stats>","logger":"task.stdout"}
{"timestamp":"2025-09-23T11:44:16.026135Z","level":"info","event":"Failure caused by Bash command failed. The command returned a non-zero exit code 1.","logger":"task.stdout"}
```
After:
```
[2025-09-24 02:12:47] INFO - Task instance is in running state
[2025-09-24 02:12:47] INFO - Previous state of the Task instance: TaskInstanceState.QUEUED
[2025-09-24 02:12:47] INFO - Current task name:get_file_stats
[2025-09-24 02:12:47] INFO - Dag name:example_bash_decorator
[2025-09-24 02:12:47] INFO - Tmp dir root location: /tmp
[2025-09-24 02:12:47] INFO - Running command: ['/usr/bin/bash', '-c', 'stat /opt/airflow/airflow-core/src/airflow/example_dags/standard/example_bash_decorator.py']
[2025-09-24 02:12:47] INFO - Output:
[2025-09-24 02:12:47] INFO - File: /opt/airflow/airflow-core/src/airflow/example_dags/standard/example_bash_decorator.py
[2025-09-24 02:12:47] INFO - Size: 3867 Blocks: 8 IO Block: 4096 regular file
[2025-09-24 02:12:47] INFO - Device: 0,34 Inode: 2057958327228072012 Links: 1
[2025-09-24 02:12:47] INFO - Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
[2025-09-24 02:12:47] INFO - Access: 2025-09-24 01:12:17.366858270 +0000
[2025-09-24 02:12:47] INFO - Modify: 2025-09-24 01:08:56.992964565 +0000
[2025-09-24 02:12:47] INFO - Change: 2025-09-24 01:08:56.992964565 +0000
[2025-09-24 02:12:47] INFO - Birth: -
[2025-09-24 02:12:47] INFO - Command exited with return code 0
[2025-09-24 02:12:47] INFO - Pushing xcom ti=RuntimeTaskInstance(id=UUID('01997947-a73f-77ea-a2d0-a4d63f57bf03'), task_id='get_file_stats', dag_id='example_bash_decorator', run_id='manual__2025-09-24T01:12:42.913441+00:00', try_number=1, dag_version_id=UUID('01997947-6324-771c-ae0e-4a01d869e171'), map_index=-1, hostname='72ace79a9422', context_carrier={}, task=<Task(_BashDecoratedOperator): get_file_stats>, bundle_instance=LocalDagBundle(name=example_dags), max_tries=0, start_date=dat [...]
[2025-09-24 02:12:47] INFO - Task instance in success state
[2025-09-24 02:12:47] INFO - Previous state of the Task instance: TaskInstanceState.RUNNING
[2025-09-24 02:12:47] INFO - Task operator:<Task(_BashDecoratedOperator): get_file_stats>
```
---
.../standard/example_dags/example_bash_decorator.py | 15 ++++-----------
1 file changed, 4 insertions(+), 11 deletions(-)
diff --git a/providers/standard/src/airflow/providers/standard/example_dags/example_bash_decorator.py b/providers/standard/src/airflow/providers/standard/example_dags/example_bash_decorator.py
index 5cdd93960cd..dfe49c52219 100644
--- a/providers/standard/src/airflow/providers/standard/example_dags/example_bash_decorator.py
+++ b/providers/standard/src/airflow/providers/standard/example_dags/example_bash_decorator.py
@@ -92,21 +92,14 @@ def example_bash_decorator():
# [END howto_decorator_bash_parametrize]
# [START howto_decorator_bash_build_cmd]
- def _get_files_in_cwd() -> list[str]:
- from pathlib import Path
-
- dir_contents = Path.cwd().glob("airflow-core/src/airflow/example_dags/*.py")
- files = [str(elem) for elem in dir_contents if elem.is_file()]
-
- return files
-
@task.bash
def get_file_stats() -> str:
+ from pathlib import Path
from shlex import join
- files = _get_files_in_cwd()
- files = files if files else ["."]
- cmd = join(["stat", *files])
+ # Get stats of the current DAG file itself
+ current_file = str(Path(__file__))
+ cmd = join(["stat", current_file])
return cmd