[GitHub] [airflow] mik-laj commented on issue #11286: Configs under `config` in values.yaml aren't applying to worker pods

[GitBox <gi...@apache.org>]

mik-laj commented on issue #11286:
URL: https://github.com/apache/airflow/issues/11286#issuecomment-705213994


   Overall, logger setup is one of the nightmares faced by people who want to start using Airflow. We have deffects in documentation, and debugging is also painful.
   I wanted to correct these situations  I added a new field to the airflow info command.
   https://github.com/apache/airflow/pull/10771
   This command will check which task handler you currently have. This is only available for Airflow 2.0, so if you want a similar effect for Airflow 1.10 you have to run python and then execute the following script.
   ```
   >>> import airflow # Initialize airflow and logger configuration
   >>> import logging
   >>> logging.getLogger('airflow.task').handlers
   [<GCSTaskHandler (NOTSET)>]
   ```
   If you see a message similar to the one below, you have a default configuration.
   `[<FileTaskHandler (NOTSET)>]` 
   also see that the `airflow config `command is not working for us, so you can also use the workaround to see the current configuration.
   ```
   from airflow import conf
   import pprint
   pprint.pprint(conf.getsection('core'))
   OrderedDict([('dags_folder', '/opt/airflow/dags/repo/dags'),
                ('base_log_folder', '/opt/airflow/logs'),
                ('remote_logging', False),
                ('remote_log_conn_id', ''),
                ('remote_base_log_folder', ''),
                ('encrypt_s3_logs', False),
                ('logging_level', 'INFO'),
                ('fab_logging_level', 'WARN'),
                ('logging_config_class', ''),
                ('colored_console_log', False),
   ```
   I use environment variables to configure and everything works fine.
   ```
       AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT = "google-cloud-platform://"
       // Configure remote logging
       // https://airflow.readthedocs.io/en/latest/logging-monitoring/logging-tasks.html#writing-logs-to-google-cloud-storage
       AIRFLOW__CORE__REMOTE_LOGGING         = "True"
       AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER = "gs://${var.gcs_logging_bucket}/"
       AIRFLOW__CORE__REMOTE_LOG_CONN_ID     = "google_cloud_default"
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org