You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Marko Kattelus (JIRA)" <ji...@apache.org> on 2019/01/23 12:30:00 UTC
[jira] [Comment Edited] (AIRFLOW-3758) Airflow command fails when
remote logging enabled to azure blob
[ https://issues.apache.org/jira/browse/AIRFLOW-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749906#comment-16749906 ]
Marko Kattelus edited comment on AIRFLOW-3758 at 1/23/19 12:29 PM:
-------------------------------------------------------------------
Configuration is done through env variables:
{code:java}
- AIRFLOW__CORE__REMOTE_LOGGING=True
- AIRFLOW__CORE__REMOTE_LOG_CONN_ID=wasb
- AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER=wasb-airflow-logs
- AIRFLOW__CORE__LOGGING_CONFIG_CLASS=log_config.LOGGING_CONFIG
{code}
I have used both with AIRFLOW__CORE__LOGGING_CONFIG_CLASS (copied from default with some minor changes) and default one without defining it.
That wasb connection is defined by connection string. Of course this connection might not exists in fresh installation yet. during running airflow initdb.
In this certain case it is already existing.
So this error is not coming when running airflow but initializing/updating db.
was (Author: kattelus):
Configuration is done through env variables:
{code}
- AIRFLOW__CORE__REMOTE_LOGGING=True
- AIRFLOW__CORE__REMOTE_LOG_CONN_ID=wasb
- AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER=wasb-airflow-logs
- AIRFLOW__CORE__LOGGING_CONFIG_CLASS=log_config.LOGGING_CONFIG
{code}
I have used both with AIRFLOW__CORE__LOGGING_CONFIG_CLASS (copied from default with some minor changes) and default one without defining it.
That was connection is defined by connection string. Of course this might not exists in fresh installation yet. during running airflow initdb.
> Airflow command fails when remote logging enabled to azure blob
> ---------------------------------------------------------------
>
> Key: AIRFLOW-3758
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3758
> Project: Apache Airflow
> Issue Type: Bug
> Components: logging
> Affects Versions: 1.10.2
> Reporter: Marko Kattelus
> Assignee: Tanay Tummalapalli
> Priority: Major
>
> I tried to update to 1.10.2 but when running
> {code:java}
> airflow initdb
> {code}
> I get following:
> {code:java}
> webserver_1 | Updating database
> webserver_1 | Unable to load the config, contains a configuration error.
> webserver_1 | Traceback (most recent call last):
> webserver_1 | File "/usr/local/lib/python3.6/logging/config.py", line 390, in resolve
> webserver_1 | found = getattr(found, frag)
> webserver_1 | AttributeError: module 'airflow.utils.log' has no attribute 'wasb_task_handler'
> webserver_1 |
> webserver_1 | During handling of the above exception, another exception occurred:
> webserver_1 |
> webserver_1 | Traceback (most recent call last):
> webserver_1 | File "/usr/local/lib/python3.6/logging/config.py", line 392, in resolve
> webserver_1 | self.importer(used)
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/utils/log/wasb_task_handler.py", line 23, in <module>
> webserver_1 | from airflow.contrib.hooks.wasb_hook import WasbHook
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/contrib/hooks/wasb_hook.py", line 22, in <module>
> webserver_1 | from airflow.hooks.base_hook import BaseHook
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 28, in <module>
> webserver_1 | from airflow.models import Connection
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/models.py", line 86, in <module>
> webserver_1 | from airflow.utils.dag_processing import list_py_file_paths
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/utils/dag_processing.py", line 49, in <module>
> webserver_1 | from airflow.settings import logging_class_path
> webserver_1 | ImportError: cannot import name 'logging_class_path'
> webserver_1 |
> webserver_1 | The above exception was the direct cause of the following exception:
> webserver_1 |
> webserver_1 | Traceback (most recent call last):
> webserver_1 | File "/usr/local/lib/python3.6/logging/config.py", line 565, in configure
> webserver_1 | handler = self.configure_handler(handlers[name])
> webserver_1 | File "/usr/local/lib/python3.6/logging/config.py", line 715, in configure_handler
> webserver_1 | klass = self.resolve(cname)
> webserver_1 | File "/usr/local/lib/python3.6/logging/config.py", line 399, in resolve
> webserver_1 | raise v
> webserver_1 | File "/usr/local/lib/python3.6/logging/config.py", line 392, in resolve
> webserver_1 | self.importer(used)
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/utils/log/wasb_task_handler.py", line 23, in <module>
> webserver_1 | from airflow.contrib.hooks.wasb_hook import WasbHook
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/contrib/hooks/wasb_hook.py", line 22, in <module>
> webserver_1 | from airflow.hooks.base_hook import BaseHook
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/hooks/base_hook.py", line 28, in <module>
> webserver_1 | from airflow.models import Connection
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/models.py", line 86, in <module>
> webserver_1 | from airflow.utils.dag_processing import list_py_file_paths
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/utils/dag_processing.py", line 49, in <module>
> webserver_1 | from airflow.settings import logging_class_path
> webserver_1 | ValueError: Cannot resolve 'airflow.utils.log.wasb_task_handler.WasbTaskHandler': cannot import name 'logging_class_path'
> webserver_1 |
> webserver_1 | During handling of the above exception, another exception occurred:
> webserver_1 |
> webserver_1 | Traceback (most recent call last):
> webserver_1 | File "/usr/local/bin/airflow", line 21, in <module>
> webserver_1 | from airflow import configuration
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/__init__.py", line 36, in <module>
> webserver_1 | from airflow import settings, configuration as conf
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/settings.py", line 262, in <module>
> webserver_1 | logging_class_path = configure_logging()
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/logging_config.py", line 72, in configure_logging
> webserver_1 | raise e
> webserver_1 | File "/usr/local/lib/python3.6/site-packages/airflow/logging_config.py", line 67, in configure_logging
> webserver_1 | dictConfig(logging_config)
> webserver_1 | File "/usr/local/lib/python3.6/logging/config.py", line 802, in dictConfig
> webserver_1 | dictConfigClass(config).configure()
> webserver_1 | File "/usr/local/lib/python3.6/logging/config.py", line 573, in configure
> webserver_1 | '%r: %s' % (name, e))
> webserver_1 | ValueError: Unable to configure handler 'processor': Cannot resolve 'airflow.utils.log.wasb_task_handler.WasbTaskHandler': cannot import name 'logging_class_path'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)