You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Niels Zeilemaker (JIRA)" <ji...@apache.org> on 2017/11/01 13:51:00 UTC

[jira] [Created] (AIRFLOW-1775) Remote file handler for logging

Niels Zeilemaker created AIRFLOW-1775:
-----------------------------------------

             Summary: Remote file handler for logging
                 Key: AIRFLOW-1775
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1775
             Project: Apache Airflow
          Issue Type: Bug
          Components: logging
            Reporter: Niels Zeilemaker
            Priority: Normal


We're using a mounted Azure file share to store our log files. Currently, Airflow is writing it's logs to that fileshare. However, this is making the solution quite expensive, as you pay per action on Azure file shares.

If we would have a remote_file_task_logging handler, we could mimic the s3_task_logging, and copy the file to the fileshare in the close method. Reducing the cost, while still providing persistent storage.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)