You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/27 13:45:59 UTC
[GitHub] [airflow] notacoat opened a new issue, #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
notacoat opened a new issue, #23967:
URL: https://github.com/apache/airflow/issues/23967
### Apache Airflow version
2.2.5
### What happened
in a worker container, a flask app is spun up to serve task log files that is read by the webserver and rendered to the user in the UI. The log files cannot be read if you overwrite the task handler's base_log_folder value. (404 error)
ie. in the airflow.cfg, the base_log_folder = `foo/bar/logs`, and the task handler uses `{base_log_folder}/dags`
### What you think should happen instead
in https://github.com/apache/airflow/blob/main/airflow/utils/serve_logs.py#L33, it should read the logging config's task handler log location.
### How to reproduce
use a custom logging config, override the task's base log folder.
Run a dag and try to view logs in the ui, you will get a 404
```
LOGGING_CONFIG["handlers"].update(
{
"task": {
"class": "airflow.utils.log.file_task_handler.FileTaskHandler",
"formatter": "airflow",
"base_log_folder": f"{BASE_LOG_FOLDER}/dags",
"filename_template": FILENAME_TEMPLATE,
"filters": ["mask_secrets"],
},
}
```
### Operating System
ubuntu
### Versions of Apache Airflow Providers
_No response_
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #23967:
URL: https://github.com/apache/airflow/issues/23967#issuecomment-1573251379
This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #23967:
URL: https://github.com/apache/airflow/issues/23967#issuecomment-1139635037
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #23967:
URL: https://github.com/apache/airflow/issues/23967#issuecomment-1616414561
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
URL: https://github.com/apache/airflow/issues/23967
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #23967:
URL: https://github.com/apache/airflow/issues/23967#issuecomment-1143429347
Feel free to provide PR for that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Owen-CH-Leung commented on issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
Posted by "Owen-CH-Leung (via GitHub)" <gi...@apache.org>.
Owen-CH-Leung commented on issue #23967:
URL: https://github.com/apache/airflow/issues/23967#issuecomment-1646552068
I can confirm that this issue still exists in the master branch.
My steps to reproduce in breeze:
1. execute `breeze start-airflow --executor CeleryExecutor --backend postgres --load-example-dags --db-reset`
2. go to `$AIRFLOW_HOME`, create a directory called `config`, and create an empty `__init__.py` and `log_config.py` file inside
3. Inside `airflow/config_templates/airflow_local_settings.py`, modify `DEFAULT_LOGGING_CONFIG[handlers][task][base_log_folder]` to some other different locations
4. copy all the contents inside `airflow/config_templates/airflow_local_settings.py` into `log_config.py`
5. In `/root/airflow/airflow.cfg` , under `logging` section, modify `logging_config_class` to `log_config.DEFAULT_LOGGING_CONFIG`
6. Restart everything
7. Run a DAG in webUI, check the log, and you will see 404 not found
The underlying reason is that the scheduler will ingest the `log_config.DEFAULT_LOGGING_CONFIG`, and outputs the log file to the location as specified by `base_log_folder` under `DEFAULT_LOGGING_CONFIG[handlers][task]`
But the flask app still attempts to serve logs from `base_log_folder` defined under the logging section (which is `/root/airflow/logs`. Hence when flask attempts to read that folder, it returned 404.
I will provide a PR to fix it soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #23967: serve_logs.py should respect the logging config's task handler's base_log_folder value
URL: https://github.com/apache/airflow/issues/23967
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org