You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/06/13 19:14:20 UTC
[GitHub] [airflow] kcphila opened a new issue, #24420: Possible configuration issue due to updated packages
kcphila opened a new issue, #24420:
URL: https://github.com/apache/airflow/issues/24420
### Apache Airflow version
2.3.2 (latest released)
### What happened
This seems odd and there may be another answer for it, but I can't find anything. We have been running 2.1.4 on Ubuntu Xenial for about a year without much trouble. So, we are switching over to the new Ubuntu 22.04 LTS environment with all up to date libraries.
When attempting to run Airflow 2.3.2, I am getting strange config-related errors when processing our custom config variable (full stack trace at the bottom).
` log_processor_filename_template = {{{{ filename }}}}.log` - This is the value included in the default template, but it fails to process with the error `jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'`
The default `log_filename_template` value also fails to process with a similar error
Commenting these out allows the config file to process, however it is not substituting templates correctly. The multiple `{AIRFLOW_HOME}` references are being interpreted literally instead of being substituted for the actual value. If I change these to reference an environment variable (e.g., `${AIRFLOW_HOME}`), it works fine.
Here's the stack trace for the log filename template. I put in a custom breakpoint to determine that it indeed is just for the log_filename_template and log_processor_filename_template variables:
```
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Unable to load the config, contains a configuration error.
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Traceback (most recent call last):
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3.10/logging/config.py", line 565, in configure
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: handler = self.configure_handler(handlers[name])
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3.10/logging/config.py", line 746, in configure_handler
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: result = factory(**kwargs)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/utils/log/file_processor_handler.py", line 44, in _init_
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: self.filename_template, self.filename_jinja_template = parse_template_string(filename_template)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/utils/helpers.py", line 176, in parse_template_string
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return None, jinja2.Template(template_string)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1195, in _new_
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return env.from_string(source, template_class=cls)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1092, in from_string
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: return cls.from_code(self, self.compile(source), gs, None)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 757, in compile
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: self.handle_exception(source=source_hint)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 925, in handle_exception
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise rewrite_traceback_stack(source=source)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "<unknown>", line 1, in template
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: The above exception was the direct cause of the following exception:
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: Traceback (most recent call last):
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/bin/airflow", line 5, in <module>
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: from airflow._main_ import main
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/_init_.py", line 47, in <module>
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: settings.initialize()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/settings.py", line 561, in initialize
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: LOGGING_CLASS_PATH = configure_logging()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/logging_config.py", line 73, in configure_logging
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise e
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/local/lib/python3.10/dist-packages/airflow/logging_config.py", line 68, in configure_logging
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: dictConfig(logging_config)
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3.10/logging/config.py", line 810, in dictConfig
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: dictConfigClass(config).configure()
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: File "/usr/lib/python3.10/logging/config.py", line 572, in configure
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: raise ValueError('Unable to configure handler '
Jun 13 18:05:08 ip-10-30-132-24 airflow[196893]: ValueError: Unable to configure handler 'processor'
```
### What you think should happen instead
Whatever is in the [official default template](https://github.com/apache/airflow/blob/2.3.2/airflow/config_templates/default_airflow.cfg) should work. I don't mind using the bash syntax in the config file, but this seems very strange.
### How to reproduce
_No response_
### Operating System
Ubuntu 22.04 AMD64 running on AWS
### Versions of Apache Airflow Providers
Airflow 2.3.2
### Deployment
Other
### Deployment details
pip-based install. I've tried this with jinja2 versions 3.1 and 3.0.3
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #24420: Possible configuration issue due to updated packages
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #24420:
URL: https://github.com/apache/airflow/issues/24420#issuecomment-1154739980
The default config template is itself a template, so `{{{{ filename }}}}.log` actually renders to `{{ filename }}.log` at runtime. This is probably the cause of your problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #24420: Possible configuration issue due to updated packages
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #24420: Possible configuration issue due to updated packages
URL: https://github.com/apache/airflow/issues/24420
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org