You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Naich (via GitHub)" <gi...@apache.org> on 2025/07/03 10:59:51 UTC
[D] SSHOperator error: TemplateNotFound [airflow]
GitHub user Naich created a discussion: SSHOperator error: TemplateNotFound
When using Airflow's SSHOperator and setting the command field to an absolute shell command (e.g., /bin/bash /home/user/test.sh), the DAG fails with a TemplateNotFound error. It appears that Airflow is treating the command string as a Jinja template file path.
Here is the error log:
> [2025-07-03, 10:48:31] INFO - DAG bundles loaded: dags-folder: source="airflow.dag_processing.bundles.manager.DagBundlesManager"
> [2025-07-03, 10:48:31] INFO - Filling up the DagBag from /home/airflow/airflow/dags/test_ssh_operator.py: source="airflow.models.dagbag.DagBag"
> [2025-07-03, 10:48:31] ERROR - Failed to resolve template field 'command': source="airflow.sdk.definitions._internal.templater"
> TemplateNotFound: '/bin/bash /home/user/test.sh' not found in search path: '/home/airflow/airflow/dags'
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/definitions/_internal/templater.py", line 98 in resolve_template_files
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/loaders.py", line 209 in get_source
>
> [2025-07-03, 10:48:31] ERROR - Exception rendering Jinja template for task 'test_ssh_operator', field 'command'. Template: '/bin/bash /home/user/test.sh': source="airflow.sdk.definitions._internal.abstractoperator"
> TemplateNotFound: '/bin/bash /home/user/test.sh' not found in search path: '/home/airflow/airflow/dags'
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/definitions/_internal/abstractoperator.py", line 379 in _do_render_template_fields
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/definitions/_internal/templater.py", line 174 in render_template
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/environment.py", line 1016 in get_template
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/environment.py", line 975 in _load_template
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/loaders.py", line 126 in load
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/loaders.py", line 209 in get_source
>
> [2025-07-03, 10:48:31] ERROR - Task failed with exception: source="task"
> TemplateNotFound: '/bin/bash /home/user/test.sh' not found in search path: '/home/airflow/airflow/dags'
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/execution_time/task_runner.py", line 832 in run
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/execution_time/task_runner.py", line 752 in _prepare
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/execution_time/task_runner.py", line 271 in render_templates
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/bases/operator.py", line 1558 in render_template_fields
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/definitions/_internal/abstractoperator.py", line 379 in _do_render_template_fields
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/airflow/sdk/definitions/_internal/templater.py", line 174 in render_template
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/environment.py", line 1016 in get_template
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/environment.py", line 975 in _load_template
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/loaders.py", line 126 in load
>
> File "/home/airflow/airflow/venv/lib/python3.11/site-packages/jinja2/loaders.py", line 209 in get_source
However, when I change the command to a simple command like "ls -l", or add some tails to the command like "/bin/bash /home/user/test.sh | tee output.log", the operator works without any issues.
I would like to know how to correctly use the SSHOperator to execute a shell script located at an absolute path on the remote server (e.g., /bin/bash /home/user/test.sh).
Is there a recommended way to pass such commands to the command field, so that Airflow does not treat them as Jinja template file paths?
Any guidance or best practices for this scenario would be greatly appreciated.
Here is my code:
```python
from airflow import DAG
from airflow.providers.ssh.operators.ssh import SSHOperator
from airflow.providers.ssh.hooks.ssh import SSHHook
with DAG(
"test_ssh_operator",
description="test ssh operator",
schedule=None,
catchup=False,
tags=["test"],
max_active_runs=1,
default_args=default_dag_args,
) as dag:
ssh_hook_monica = SSHHook(
ssh_conn_id=None,
remote_host=IP,
port=22,
username="USER"
)
sync_cmd = "/bin/bash /home/user/test.sh"
test = SSHOperator(
task_id="test_ssh_operator",
command=sync_cmd,
ssh_hook=ssh_hook_monica,
conn_timeout=60,
cmd_timeout=3600,
)
test
```
My environment:
```
airflow@workplace:~/airflow$ pip freeze|grep airflow
apache-airflow==3.0.1
apache-airflow-core==3.0.1
apache-airflow-providers-amazon==9.8.0
apache-airflow-providers-celery==3.10.6
apache-airflow-providers-common-compat==1.6.1
apache-airflow-providers-common-io==1.5.4
apache-airflow-providers-common-sql==1.27.0
apache-airflow-providers-fab==2.0.2
apache-airflow-providers-http==5.3.0
apache-airflow-providers-smtp==2.0.3
apache-airflow-providers-ssh==4.1.0
apache-airflow-providers-standard==1.1.0
apache-airflow-task-sdk==1.0.1
```
GitHub link: https://github.com/apache/airflow/discussions/52767
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] SSHOperator error: TemplateNotFound [airflow]
Posted by "Naich (via GitHub)" <gi...@apache.org>.
GitHub user Naich added a comment to the discussion: SSHOperator error: TemplateNotFound
Thanks for the suggestion!
literal works. In Airflow 3.0.x should could be imported from :
`from airflow.sdk.definitions.template import literal`
GitHub link: https://github.com/apache/airflow/discussions/52767#discussioncomment-13839984
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] SSHOperator error: TemplateNotFound [airflow]
Posted by "rahidzeynal (via GitHub)" <gi...@apache.org>.
GitHub user rahidzeynal added a comment to the discussion: SSHOperator error: TemplateNotFound
Here's a professional and concise comment you can post in the GitHub issue to share your findings and workaround for the Airflow `SSHOperator` templating issue:
---
I faced the exact same issue with Airflow **2.7.2**, where passing an absolute shell command like `/bin/bash /home/user/test.sh` to `SSHOperator` caused a `TemplateNotFound` error. It seems that Airflow was trying to treat the entire command string as a Jinja template path.
### Root Cause:
The `command` field in `SSHOperator` is templated by default. So if the command string resembles a file path, Jinja tries to load it as a template from the DAGs folder, which leads to the `TemplateNotFound` error.
### Workaround / Solution:
To bypass this templating behavior, I subclassed `SSHOperator` and disabled templating for the `command` field:
```python
from airflow.providers.ssh.operators.ssh import SSHOperator
class NoTemplateSSHOperator(SSHOperator):
template_fields = () # Disable all templating for this operator
```
Then used it like this:
```python
execute_SAP = NoTemplateSSHOperator(
task_id='submit_sh',
ssh_conn_id='your_ssh_conn_id',
command='/bin/bash /path/to/sh/file/sample_sh_echo.sh',
conn_timeout=2400,
cmd_timeout=2400,
)
```
This effectively prevents Airflow from interpreting the command as a template path.
---
Starting from Airflow 2.8, there's a better way to prevent templating without modifying the operator — by wrapping the string with the built-in literal() function.
```python
from airflow.utils.template import literal
execute_SAP = SSHOperator(
task_id='submit_sh',
ssh_conn_id='your_ssh_conn_id',
command=literal('/bin/bash /path/to/sh/file/sample_sh_echo.sh'),
conn_timeout=2400,
cmd_timeout=2400,
)
```
This approach is cleaner and avoids the need for subclassing.
---
GitHub link: https://github.com/apache/airflow/discussions/52767#discussioncomment-13837280
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org