You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/07 10:06:27 UTC
[GitHub] [airflow] ecerulm opened a new issue #12876: Improve error message when template fields contain invalid entries
ecerulm opened a new issue #12876:
URL: https://github.com/apache/airflow/issues/12876
** Description **
When a new operator is defined there are some issues with `template_fields` that produces somewhat misleading error messages
* `template_fields = ('myfield')`, which of course is incorrect because that is not a tuple, you need to use `('myfield',)` or `['myfield']`
* `template_field = ['field_with_a_typo']`
As a I said both gives errors (at different stages) but the errors are a bit cryptic
For example:
```
[2020-12-07 09:50:45,088] {taskinstance.py:1150} ERROR - 'SFTPToS3Operator2' object has no attribute 's3_key'
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 965, in _run_raw_task
self.render_templates(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1424, in render_templates
self.task.render_template_fields(context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 719, in render_template_fields
self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 724, in _do_render_template_fields
content = getattr(parent, attr_name)
AttributeError: 'SFTPToS3Operator2' object has no attribute 's3_key'
```
where there is no mention that this is related to `templated_fields`
**Use case / motivation**
In order to have a better experience developing plugins I would like
* A warning / error if a str is used for `template_fields = 'myfield'`. It's very unlikely that anyone want to use `myfield` as sequence `['m','y', 'f','i','e','l','d']`.
* A more specific error message in `_run_raw_task` if `template_fields` contains attributes not present.
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
**Description**
<!-- A short description of your feature -->
**Use case / motivation**
<!-- What do you want to happen?
Rather than telling us how you might implement this solution, try to take a
step back and describe what you are trying to achieve.
-->
**Related Issues**
<!-- Is there currently another issue associated with this? -->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] baolsen commented on issue #12876: Improve error message when template fields contain invalid entries
Posted by GitBox <gi...@apache.org>.
baolsen commented on issue #12876:
URL: https://github.com/apache/airflow/issues/12876#issuecomment-853614486
Related https://stackoverflow.com/a/58748028/2409299
I agree with the suggestion to emit a warning if str is specified, along with wrapping the string in a list automatically to save some headaches like these.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #12876: Improve error message when template fields contain invalid entries
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #12876:
URL: https://github.com/apache/airflow/issues/12876
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] baolsen commented on issue #12876: Improve error message when template fields contain invalid entries
Posted by GitBox <gi...@apache.org>.
baolsen commented on issue #12876:
URL: https://github.com/apache/airflow/issues/12876#issuecomment-853614486
Related https://stackoverflow.com/a/58748028/2409299
I agree with the suggestion to emit a warning if str is specified, along with wrapping the string in a list automatically to save some headaches like these.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #12876: Improve error message when template fields contain invalid entries
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #12876:
URL: https://github.com/apache/airflow/issues/12876#issuecomment-853620579
Or maybe just magically convert it to a `list[str]`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #12876: Improve error message when template fields contain invalid entries
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #12876:
URL: https://github.com/apache/airflow/issues/12876#issuecomment-853620579
Or maybe just magically convert it to a `list[str]`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org