[PR] Return no log lines when the ECS task's log stream is missing [airflow]

["fpiped (via GitHub)" <gi...@apache.org>]

fpiped opened a new pull request, #74744:
URL: https://github.com/apache/airflow/pull/74744

   `AwsTaskLogFetcher.get_last_log_messages` called `GetLogEvents` without handling a missing stream. When `EcsRunTaskOperator` cannot find the task's log stream (for example with a wrong `awslogs_stream_prefix`), a non-zero exit then failed with `ResourceNotFoundException` instead of the operator's own error, and `skip_on_exit_code` was ignored. A missing stream now gives no log lines and a warning naming the group and stream, as the fetcher's polling loop already does; other errors still raise. This also covers `get_last_log_message`, which the operator uses for the XCom of a successful task. `BatchOperator` already catches errors around the same call.
   
   closes: #74699
   
   Testing:
   
   - `test_task_log_fetcher.py`: without a log stream, `get_last_log_messages` returns `[]` and `get_last_log_message` `None`, with the warning; another `ClientError` still raises.
   - `test_ecs.py`: with a real `AwsTaskLogFetcher` whose `GetLogEvents` fails with `ResourceNotFoundException`, an exit code in `skip_on_exit_code` raises `AirflowSkipException`.
   - Both files: 124 passed (121 on `main` plus the 3 new tests). Without the change in `task_log_fetcher.py`, the two missing-stream tests fail; the other-errors test passes before and after. Five `test_ecs.py` tests error locally on `main` too (`No module named 'airflow_shared'`).
   - `ruff check` and `ruff format --check` (0.16.8).
   - On ECS Fargate with Airflow 3.3.2 and this branch: a container that exits 3 with `skip_on_exit_code=[3]` and a stream prefix that does not match is now skipped, with the warning; with the right prefix the behaviour is unchanged.
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Return no log lines when the ECS task's log stream is missing [airflow]

["fpiped (via GitHub)" <gi...@apache.org>]

fpiped closed pull request #74744: Return no log lines when the ECS task's log stream is missing
URL: https://github.com/apache/airflow/pull/74744


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Return no log lines when the ECS task's log stream is missing [airflow]

["fpiped (via GitHub)" <gi...@apache.org>]

fpiped commented on PR #74744:
URL: https://github.com/apache/airflow/pull/74744#issuecomment-6466854186

   Closing: #74730 landed the same fix (a missing log stream now gives no log lines, so the operator raises its own error or skips) and has been merged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org