You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/11/24 23:40:57 UTC

[GitHub] [airflow] virendhar opened a new issue #19817: ECS Operator: Capture container execution logs in Airflow task execution log

virendhar opened a new issue #19817:
URL: https://github.com/apache/airflow/issues/19817


   ### Description
   
   ## Objective:
   
   Add functionality to Airflow ECSOperator to capture the task execution logs as they happen from AWS CloudWatch to the ECSOperator's task execution logs.
   
   ### Use case/motivation
   
   ## Present Challenge: 
   When executing Fargate tasks via ECSOperator the logs are not readily available in Airflow task during and after execution. The final status of the container execution is reflected in Airflow ECS Operator task logs. As a result one should navigate the task logs in Airflow to get a handle of the container log path in AWS Cloudwatch and then access the log from AWS Cloudwatch. 
   
   The current behavior introduces the following challenges,
   - The ongoing/final execution logs are not captured in Airflow task ( ECS Operator ) log. This typically involves couple of additional steps - which can be avoided.
   - The visibility is limited to the final status of the task execution
   
   ## Solution
   
   Mechanism to relay the logs from AWS CloudWatch during execution to Airflow logs as and when they are happening.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #19817: ECS Operator: Capture container execution logs in Airflow task execution log

Posted by GitBox <gi...@apache.org>.
potiuk closed issue #19817:
URL: https://github.com/apache/airflow/issues/19817


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #19817: ECS Operator: Capture container execution logs in Airflow task execution log

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #19817:
URL: https://github.com/apache/airflow/issues/19817#issuecomment-978470913


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] virendhar-aws commented on issue #19817: ECS Operator: Capture container execution logs in Airflow task execution log

Posted by GitBox <gi...@apache.org>.
virendhar-aws commented on issue #19817:
URL: https://github.com/apache/airflow/issues/19817#issuecomment-1019651987


   This ticket can be closed. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org