You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/08 14:08:40 UTC

[GitHub] [airflow] sshah90 opened a new issue #12920: Integration with Grafana Loki 

sshah90 opened a new issue #12920:
URL: https://github.com/apache/airflow/issues/12920


   **Description**
   
   Grafana [Loki](https://grafana.com/oss/loki/) is an emerging Open Source project. Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost-effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream.
   
   Currently, Airflow has integration with different stacks (ex. S3, Azure blob) for logging solutions but none of them provide a cloud-agnostic real-time logging platform.
   
   **Use case / motivation**
   
   Create a log handler that pushes task logs to Loki with labels and using the same labels pull logs to Airflow Web UI. 
   
   Loki Loghandler should add these features
   
   - Get real-time log output in the UI
   - Centralize all Airflow logging so that logs could be viewed at a "DAG level" (ie. multiple task logs in the same context) using Grafana
   
   **Related Issues**
   None


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] sgrzemski commented on issue #12920: Integration with Grafana Loki

Posted by GitBox <gi...@apache.org>.
sgrzemski commented on issue #12920:
URL: https://github.com/apache/airflow/issues/12920#issuecomment-811041893


   That would be great!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] yoavo-datricks commented on issue #12920: Integration with Grafana Loki

Posted by GitBox <gi...@apache.org>.
yoavo-datricks commented on issue #12920:
URL: https://github.com/apache/airflow/issues/12920#issuecomment-853839802


   Hi. Any news on this? This integration is very useful these days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #12920: Integration with Grafana Loki

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12920:
URL: https://github.com/apache/airflow/issues/12920#issuecomment-855274019


   I think if someone wants to contribute, it would be great. Maybe even grafana provider could be developed by Grafana Labs? We have a number of providers developed and controlled by third parties (for example Great Expectations provider https://github.com/great-expectations/airflow-provider-great-expectations ). Happy to help someone from Grafana Labs to do so :).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #12920: Integration with Grafana Loki

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12920:
URL: https://github.com/apache/airflow/issues/12920#issuecomment-740640846


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] yoavo-datricks commented on issue #12920: Integration with Grafana Loki

Posted by GitBox <gi...@apache.org>.
yoavo-datricks commented on issue #12920:
URL: https://github.com/apache/airflow/issues/12920#issuecomment-853839802


   Hi. Any news on this? This integration is very useful these days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org