You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/27 02:14:24 UTC

[GitHub] [airflow] mik-laj opened a new issue #10593: Missing docs for logging

mik-laj opened a new issue #10593:
URL: https://github.com/apache/airflow/issues/10593


   Hello,
   
   The configuration of the Airflow logger is not easy, as in most applications. We have many loggers, some loggers generate multiple streams and our architecture is heterogeneous, so we end up with multiple log streams. This is very confusing for many users, and the lack of documentation doesn't make it any easier.
   
   We have following loggers
    - ``root logger``
    - ``airflow.processor``
    - ``airflow.task``
    - ``flask_appbuilder``
    - ``airflow.processor_manager``
   
   ``airflow.task`` creates a new log file for each pair of task instances and try numbers. These entries are accessible via the Web UI.
   ``airflow.processor`` creates a new log file for each DAG File.
   ``flask_appbuilder`` is only used to filter logs records because the FAB is too verbose, so in most cases, we don't have to configure it specially. 
   
   Webserver writes logs to ``root logger``.
   Worker writes logs to ``airflow.processor`` and ``root logger``
   Scheduler writes logs to ``airflow.processor``, ``airflow.processor_manager`` and ``root logger``
   
   I would be happy if there was a detailed description for the default configuration and the methods on how to influence these configurations were described. Currently, the documentation only describes configurations for ``airflow.task``.   I wonder if I have the correct configuration for ``airflow.processor``. in my environment. The community only supports writing logs from this logger to a files for each DAG file, but this is not a useful solution in an environment with volatile volumes like Kubernetes.
   
   Best regards,
   Kamil BreguĊ‚a
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on issue #10593: Missing docs for logging

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #10593:
URL: https://github.com/apache/airflow/issues/10593#issuecomment-1003123808


   cc @blag 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org