You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jgoedeke (via GitHub)" <gi...@apache.org> on 2025/09/09 09:34:34 UTC
[I] Airflow 3.x templating calls resolve() on Path-like objects with context argument, causing FileNotFoundError in custom subclasses [airflow]
jgoedeke opened a new issue, #55412:
URL: https://github.com/apache/airflow/issues/55412
### Apache Airflow version
3.0.6
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
When using custom classes that subclass pathlib.PosixPath, Airflow 3's new SDK templating system calls .resolve(context) on these objects when rendering templates. If the subclass doesn't override resolve to accept a context argument, the method inherited from PosixPath is called, and the context (a dict) is interpreted as 'strict=True', causing Path.resolve(strict=True) and triggering FileNotFoundError if the file doesn't exist. This is a regression from Airflow 2.x and breaks compatibility with custom path-like objects in op_kwargs and other templated fields.
Example error:
```
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/some/file.txt'
File "/home/airflow/.local/lib/python3.11/site-packages/airflow/sdk/definitions/_internal/templater.py", line 192 in render_template
File "/home/airflow/.local/lib/python3.11/site-packages/custom_file_lib/path.py", line 140 in resolve
File "/usr/local/lib/python3.11/pathlib.py", line 993 in resolve
```
This behavior can break any custom library that extends pathlib.Path and doesn't override resolve(context).
### What you think should happen instead?
The templater should not call .resolve(context) on objects that inherit from pathlib.Path, or should handle cases where resolve is not designed for templating. Ideally, the templater should distinguish between 'templating resolution' and 'filesystem resolution'. Or, provide a clear interface so custom classes can be compatible without breaking existing path-like subclasses.
### How to reproduce
1. Create a custom class that subclasses pathlib.PosixPath and does not override resolve().
2. Pass an instance of this class as a templated field/op_kwargs in Airflow 3.x SDK DAG.
3. Trigger task rendering (e.g. via BashOperator or similar).
4. Observe FileNotFoundError during template rendering if the file does not exist.
### Operating System
Linux (Debian-based container)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other Docker-based deployment
### Deployment details
_No response_
### Anything else?
Happens every time a custom Path subclass is passed to a templated field and the file does not exist yet.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.x templating calls resolve() on Path-like objects with context argument, causing FileNotFoundError in custom subclasses [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #55412: Airflow 3.x templating calls resolve() on Path-like objects with context argument, causing FileNotFoundError in custom subclasses
URL: https://github.com/apache/airflow/issues/55412
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.x templating calls resolve() on Path-like objects with context argument, causing FileNotFoundError in custom subclasses [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55412:
URL: https://github.com/apache/airflow/issues/55412#issuecomment-3270773457
@vikrantkumar-max, please feel free to open up a PR for this issue. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.x templating calls resolve() on Path-like objects with context argument, causing FileNotFoundError in custom subclasses [airflow]
Posted by "vikrantkumar-max (via GitHub)" <gi...@apache.org>.
vikrantkumar-max commented on issue #55412:
URL: https://github.com/apache/airflow/issues/55412#issuecomment-3269880291
I would Like to work on this ticket
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org