You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Limess (via GitHub)" <gi...@apache.org> on 2023/05/02 12:08:05 UTC
[GitHub] [airflow] Limess opened a new issue, #31014: Exception when manually triggering dags via UI
Limess opened a new issue, #31014:
URL: https://github.com/apache/airflow/issues/31014
### Apache Airflow version
2.6.0
### What happened
When clicking the "Trigger DAG w/ config" in a DAG UI I receive a 500 "Oops" page.
The Airflow webserver logs show this:
```
2023-05-02T13:02:50 - [2023-05-02T12:02:50.249+0000] {app.py:1744} ERROR - Exception on /trigger [GET]
2023-05-02T13:02:50 - Traceback (most recent call last):
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask/app.py", line 2529, in wsgi_app
2023-05-02T13:02:50 - response = self.full_dispatch_request()
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask/app.py", line 1825, in full_dispatch_request
2023-05-02T13:02:50 - rv = self.handle_user_exception(e)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
2023-05-02T13:02:50 - rv = self.dispatch_request()
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
2023-05-02T13:02:50 - return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/www/auth.py", line 47, in decorated
2023-05-02T13:02:50 - return func(*args, **kwargs)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/www/decorators.py", line 125, in wrapper
2023-05-02T13:02:50 - return f(*args, **kwargs)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/utils/session.py", line 76, in wrapper
2023-05-02T13:02:50 - return func(*args, session=session, **kwargs)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/www/views.py", line 1967, in trigger
2023-05-02T13:02:50 - return self.render_template(
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/www/views.py", line 640, in render_template
2023-05-02T13:02:50 - return super().render_template(
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask_appbuilder/baseviews.py", line 339, in render_template
2023-05-02T13:02:50 - return render_template(
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask/templating.py", line 147, in render_template
2023-05-02T13:02:50 - return _render(app, template, context)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask/templating.py", line 130, in _render
2023-05-02T13:02:50 - rv = template.render(context)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/jinja2/environment.py", line 1301, in render
2023-05-02T13:02:50 - self.environment.handle_exception()
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/jinja2/environment.py", line 936, in handle_exception
2023-05-02T13:02:50 - raise rewrite_traceback_stack(source=source)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/www/templates/airflow/trigger.html", line 106, in top-level template code
2023-05-02T13:02:50 - <span class="help-block">{{ form_details.description }}</span>
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/www/templates/airflow/main.html", line 21, in top-level template code
2023-05-02T13:02:50 - {% from 'airflow/_messages.html' import show_message %}
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html", line 2, in top-level template code
2023-05-02T13:02:50 - {% import 'appbuilder/baselib.html' as baselib %}
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask_appbuilder/templates/appbuilder/init.html", line 42, in top-level template code
2023-05-02T13:02:50 - {% block body %}
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/flask_appbuilder/templates/appbuilder/baselayout.html", line 19, in block 'body'
2023-05-02T13:02:50 - {% block content %}
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/www/templates/airflow/trigger.html", line 162, in block 'content'
2023-05-02T13:02:50 - {{ form_element(form_key, form_details) }}
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/jinja2/runtime.py", line 777, in _invoke
2023-05-02T13:02:50 - rv = self._func(*arguments)
2023-05-02T13:02:50 - File "/opt/airflow/.local/lib/python3.10/site-packages/airflow/www/templates/airflow/trigger.html", line 83, in template
2023-05-02T13:02:50 - {%- for txt in form_details.value -%}
2023-05-02T13:02:50 - TypeError: 'NoneType' object is not iterable
```
### What you think should happen instead
No error is shown (worked in 2.5.2)
### How to reproduce
Create a DAG with the following config defined for parameters:
```
params={
"delete_actions": Param(
False,
description="Whether to delete actions after execution.",
type="boolean",
),
"dates": Param(
None,
description="An explicit list of date strings to run on.",
type=["null", "array"],
minItems=1,
),
"start_date_inclusive": Param(
None,
description="An inclusive start-date used to generate a list of dates to run on.",
type=["null", "string"],
pattern="^[0-9]{4}[-/][0-9]{2}[-/][0-9]{2}$",
),
"end_date_exclusive": Param(
None,
description="An exclusive end-date used to generate a list of dates to run on.",
type=["null", "string"],
pattern="^[0-9]{4}[-/][0-9]{2}[-/][0-9]{2}$",
),
"actions_bucket_name": Param(
None,
description='An S3 bucket to read batch actions from. Set as "ACTIONS_BUCKET".',
type=["null", "string"],
),
"actions_path_prefix": Param(
None,
description='An S3 bucket to read batch actions from. Prefixes "ACTIONS_PATH".',
type=["null", "string"],
pattern="^.+/$",
),
"sns_output_topic_name": Param(
None,
description='An SNS output topic ARN to set as "DATA_READY_TO_INDEX_OUTPUT_TOPIC."',
type=["null", "string"],
),
},
# required to convert params to their correct types
render_template_as_native_obj=True,
```
Deploy the DAG, click the manual trigger button.
### Operating System
Debian
### Versions of Apache Airflow Providers
N/A
### Deployment
Other Docker-based deployment
### Deployment details
Amazon ECS
Python version: 3.10.11
Airflow version: 2.6.0 (official docker image as base)
### Anything else
Occurs every time.
Does *NOT* occur when `params` are not defined on the DAG.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Limess commented on issue #31014: Exception when manually triggering dags via UI with `params` defined.
Posted by "Limess (via GitHub)" <gi...@apache.org>.
Limess commented on issue #31014:
URL: https://github.com/apache/airflow/issues/31014#issuecomment-1531390225
When I try to simplify it and change to `type="array"` I get other errors, not sure what the correct syntax is for an optional list param:
```
Broken DAG: [/opt/airflow/dags/signal_ai/data_platform/articles/offline_processing_articles_update_chunked_store.py] ParamValidationError: Invalid input for param dates: None is not of type 'array'
Failed validating 'type' in schema:
{'type': 'array'}
On instance:
None
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Limess commented on issue #31014: Exception when manually triggering dags via UI with `params` defined.
Posted by "Limess (via GitHub)" <gi...@apache.org>.
Limess commented on issue #31014:
URL: https://github.com/apache/airflow/issues/31014#issuecomment-1531393797
```
Param(
[],
description="An explicit list of date strings to run on.",
type=["null", "array"],
minItems=0,
),
```
resolves this and seems to be what I want, with a slight change of behaviour (results in an empty list rather than None when unset).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #31014: Exception when manually triggering dags via UI with `params` defined.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #31014:
URL: https://github.com/apache/airflow/issues/31014#issuecomment-1531505836
Marked it as good-first-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] herlambang commented on issue #31014: Exception when manually triggering dags via UI with `params` defined.
Posted by "herlambang (via GitHub)" <gi...@apache.org>.
herlambang commented on issue #31014:
URL: https://github.com/apache/airflow/issues/31014#issuecomment-1532386183
I also noticed on this page https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/params.html#use-params-to-provide-a-trigger-ui-form, under `If you use the Param class as definition of the param value, the following parameters can be added`
```
list: Generates a HTML multi line text field, every line edited will be made into a string array as value
```
`list:` should be `array:` instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Limess commented on issue #31014: Exception when manually triggering dags via UI with `params` defined.
Posted by "Limess (via GitHub)" <gi...@apache.org>.
Limess commented on issue #31014:
URL: https://github.com/apache/airflow/issues/31014#issuecomment-1531364101
I've reduced this down to the param causing the issue:
```
"dates": Param(
None,
description="An explicit list of date strings to run on.",
type=["null", "array"],
minItems=1,
),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] herlambang commented on issue #31014: Exception when manually triggering dags via UI with `params` defined.
Posted by "herlambang (via GitHub)" <gi...@apache.org>.
herlambang commented on issue #31014:
URL: https://github.com/apache/airflow/issues/31014#issuecomment-1532384488
I confirm this happened on development branch as well, I can work on this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #31014: Exception when manually triggering dags via UI with `params` defined.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #31014:
URL: https://github.com/apache/airflow/issues/31014#issuecomment-1531505316
YEah. I think it's not a high-priority ones, but should be solved :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #31014: Exception when manually triggering dags via UI with `params` defined.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #31014: Exception when manually triggering dags via UI with `params` defined.
URL: https://github.com/apache/airflow/issues/31014
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org