You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/04 11:23:45 UTC
[GitHub] [airflow] potiuk opened a new pull request #17410: Improve diagnostics message when users have secret_key misconfigured
potiuk opened a new pull request #17410:
URL: https://github.com/apache/airflow/pull/17410
Recently fixed log open-access vulnerability have caused
quite a lot of questions and issues from the affected users who
did not have webserver/secret_key configured for their workers
(effectively leading to random value for those keys for workers)
This PR explicitly explains the possible reason for the problem and
encourages the user to configure their webserver's secret_key
in both - workers and webserver.
Related to: #17251 and a number of similar slack discussions.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #17410:
URL: https://github.com/apache/airflow/pull/17410#discussion_r682528405
##########
File path: airflow/utils/log/file_task_handler.py
##########
@@ -190,6 +191,15 @@ def _read(self, ti, try_number, metadata=None):
response.raise_for_status()
log += '\n' + response.text
+ except HTTPStatusError as e:
+ if e.response.status_code == 403:
+ log += f"*** Failed to fetch log file from worker With 403, Forbidden error. {str(e)}\n"
Review comment:
I put it a bit differently actually - before raising the status to make it simpler. I think it's still worth to add the str(err) in next line anyway as it might contain potentially some more diagnostics messages (for example if the forbidden is returned rather by a proxy than worker - it might contain extra message).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #17410:
URL: https://github.com/apache/airflow/pull/17410#discussion_r682526692
##########
File path: airflow/utils/log/file_task_handler.py
##########
@@ -190,6 +191,15 @@ def _read(self, ti, try_number, metadata=None):
response.raise_for_status()
log += '\n' + response.text
+ except HTTPStatusError as e:
+ if e.response.status_code == 403:
+ log += f"*** Failed to fetch log file from worker With 403, Forbidden error. {str(e)}\n"
Review comment:
```suggestion
log += f"*** Failed to fetch log file from worker With {str(e)}\n"
```
I'd guess (but haven't tested) that stringifying the error would show "403 Forbidden" -- and if so we don't need to say that again.
##########
File path: airflow/utils/log/file_task_handler.py
##########
@@ -22,6 +22,7 @@
from typing import TYPE_CHECKING, Optional
import httpx
+from httpx import HTTPStatusError
Review comment:
```suggestion
```
Unused import now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #17410:
URL: https://github.com/apache/airflow/pull/17410
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #17410:
URL: https://github.com/apache/airflow/pull/17410#discussion_r682526692
##########
File path: airflow/utils/log/file_task_handler.py
##########
@@ -190,6 +191,15 @@ def _read(self, ti, try_number, metadata=None):
response.raise_for_status()
log += '\n' + response.text
+ except HTTPStatusError as e:
+ if e.response.status_code == 403:
+ log += f"*** Failed to fetch log file from worker With 403, Forbidden error. {str(e)}\n"
Review comment:
```suggestion
log += f"*** Failed to fetch log file from worker With {str(e)}\n"
```
I'd guess (but haven't tested) that stringifying the error would show "403 Forbidden" -- and if so we don't need to say that again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #17410:
URL: https://github.com/apache/airflow/pull/17410#discussion_r682534238
##########
File path: airflow/utils/log/file_task_handler.py
##########
@@ -22,6 +22,7 @@
from typing import TYPE_CHECKING, Optional
import httpx
+from httpx import HTTPStatusError
Review comment:
```suggestion
```
Unused import now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #17410:
URL: https://github.com/apache/airflow/pull/17410#issuecomment-892588632
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #17410:
URL: https://github.com/apache/airflow/pull/17410#discussion_r682528405
##########
File path: airflow/utils/log/file_task_handler.py
##########
@@ -190,6 +191,15 @@ def _read(self, ti, try_number, metadata=None):
response.raise_for_status()
log += '\n' + response.text
+ except HTTPStatusError as e:
+ if e.response.status_code == 403:
+ log += f"*** Failed to fetch log file from worker With 403, Forbidden error. {str(e)}\n"
Review comment:
I put it a bit differently actually - before raising the status to make it simpler. I think it's still worth to add the str(err) in next line anyway as it might contain potentially some more diagnostics messages (for example if the forbidden is returned rather by a proxy than worker - it might contain extra message).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #17410:
URL: https://github.com/apache/airflow/pull/17410
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #17410: Improve diagnostics message when users have secret_key misconfigured
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #17410:
URL: https://github.com/apache/airflow/pull/17410#issuecomment-892588632
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org