You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2024/04/04 10:27:41 UTC
[PR] Make UV_HTTP_TIMEOUT default value bigger in WSL2 environment [airflow]
potiuk opened a new pull request, #38742:
URL: https://github.com/apache/airflow/pull/38742
When uv docker build is run in WSL2 the default timeout does not cope well with downloading big packages and times-out with the default UV timeout. This PR performs detection of whether you are in a WSL environment and changes the timeout in case you are in WSL2. It also raises error when WSL1 is detected (not supported by Breeze at all). In case the WSL1 detection is wrong we also have an escape hatch by manually specifying `--uv-http-timeout`.
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make UV_HTTP_TIMEOUT default value bigger in WSL2 environment [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #38742:
URL: https://github.com/apache/airflow/pull/38742#discussion_r1551446250
##########
dev/breeze/src/airflow_breeze/utils/platforms.py:
##########
@@ -16,10 +16,69 @@
# under the License.
from __future__ import annotations
+import platform
+import sys
+from pathlib import Path
+
def get_real_platform(single_platform: str) -> str:
"""
Replace different platform variants of the platform provided platforms with the two canonical ones we
are using: amd64 and arm64.
"""
return single_platform.replace("x86_64", "amd64").replace("aarch64", "arm64").replace("/", "-")
+
+
+def _exists_no_permission_error(p: str) -> bool:
+ try:
+ return Path(p).exists()
+ except PermissionError:
+ return False
+
+
+def message_on_wsl1_detected(release_name: str | None, kernel_version: tuple[int, ...] | None):
+ from airflow_breeze.utils.console import get_console
+
+ get_console().print("[error]You are running WSL1 - Breeze requires WSL2! Quitting.\n")
+ get_console().print("[warning]It can also be that our detection mechanism is wrong:[/]\n\n")
+ if release_name:
+ get_console().print(f"[info]We based our WSL1 detection on the release name: `{release_name}`\n")
+ elif kernel_version:
+ get_console().print(f"[info]We based our WSL1 detection on the kernel version: `{kernel_version}`\n")
+ get_console().print(
+ "[info]If you are running WSL2, please report this issue to the maintainers\n"
+ "of Airflow, so we can improve the detection mechanism.\n"
+ "You can also try to run the command with `--uv-http-timeout 900` or "
+ "`--no-use-uv` flag to skip the WSL1 check.\n"
+ )
+
+
+def is_wsl2() -> bool:
+ """
+ Check if the current platform is WSL
+ :return: True if the current platform is WSL
+ """
+ release_name = platform.uname().release
Review Comment:
Why not :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make UV_HTTP_TIMEOUT default value bigger in WSL2 environment [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #38742:
URL: https://github.com/apache/airflow/pull/38742
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make UV_HTTP_TIMEOUT default value bigger in WSL2 environment [airflow]
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on code in PR #38742:
URL: https://github.com/apache/airflow/pull/38742#discussion_r1551441913
##########
dev/breeze/src/airflow_breeze/utils/platforms.py:
##########
@@ -16,10 +16,69 @@
# under the License.
from __future__ import annotations
+import platform
+import sys
+from pathlib import Path
+
def get_real_platform(single_platform: str) -> str:
"""
Replace different platform variants of the platform provided platforms with the two canonical ones we
are using: amd64 and arm64.
"""
return single_platform.replace("x86_64", "amd64").replace("aarch64", "arm64").replace("/", "-")
+
+
+def _exists_no_permission_error(p: str) -> bool:
+ try:
+ return Path(p).exists()
+ except PermissionError:
+ return False
+
+
+def message_on_wsl1_detected(release_name: str | None, kernel_version: tuple[int, ...] | None):
+ from airflow_breeze.utils.console import get_console
+
+ get_console().print("[error]You are running WSL1 - Breeze requires WSL2! Quitting.\n")
+ get_console().print("[warning]It can also be that our detection mechanism is wrong:[/]\n\n")
+ if release_name:
+ get_console().print(f"[info]We based our WSL1 detection on the release name: `{release_name}`\n")
+ elif kernel_version:
+ get_console().print(f"[info]We based our WSL1 detection on the kernel version: `{kernel_version}`\n")
+ get_console().print(
+ "[info]If you are running WSL2, please report this issue to the maintainers\n"
+ "of Airflow, so we can improve the detection mechanism.\n"
+ "You can also try to run the command with `--uv-http-timeout 900` or "
+ "`--no-use-uv` flag to skip the WSL1 check.\n"
+ )
+
+
+def is_wsl2() -> bool:
+ """
+ Check if the current platform is WSL
+ :return: True if the current platform is WSL
+ """
+ release_name = platform.uname().release
Review Comment:
If it not a linux we could stop check in very beginning
```suggestion
if not sys.platform.startswith("linux"):
return False
release_name = platform.uname().release
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make UV_HTTP_TIMEOUT default value bigger in WSL2 environment [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #38742:
URL: https://github.com/apache/airflow/pull/38742#issuecomment-2036792900
cc: @dabla
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make UV_HTTP_TIMEOUT default value bigger in WSL2 environment [airflow]
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on code in PR #38742:
URL: https://github.com/apache/airflow/pull/38742#discussion_r1551452922
##########
dev/breeze/src/airflow_breeze/utils/platforms.py:
##########
@@ -16,10 +16,69 @@
# under the License.
from __future__ import annotations
+import platform
+import sys
+from pathlib import Path
+
def get_real_platform(single_platform: str) -> str:
"""
Replace different platform variants of the platform provided platforms with the two canonical ones we
are using: amd64 and arm64.
"""
return single_platform.replace("x86_64", "amd64").replace("aarch64", "arm64").replace("/", "-")
+
+
+def _exists_no_permission_error(p: str) -> bool:
+ try:
+ return Path(p).exists()
+ except PermissionError:
+ return False
+
+
+def message_on_wsl1_detected(release_name: str | None, kernel_version: tuple[int, ...] | None):
+ from airflow_breeze.utils.console import get_console
+
+ get_console().print("[error]You are running WSL1 - Breeze requires WSL2! Quitting.\n")
+ get_console().print("[warning]It can also be that our detection mechanism is wrong:[/]\n\n")
+ if release_name:
+ get_console().print(f"[info]We based our WSL1 detection on the release name: `{release_name}`\n")
+ elif kernel_version:
+ get_console().print(f"[info]We based our WSL1 detection on the kernel version: `{kernel_version}`\n")
+ get_console().print(
+ "[info]If you are running WSL2, please report this issue to the maintainers\n"
+ "of Airflow, so we can improve the detection mechanism.\n"
+ "You can also try to run the command with `--uv-http-timeout 900` or "
+ "`--no-use-uv` flag to skip the WSL1 check.\n"
+ )
+
+
+def is_wsl2() -> bool:
+ """
+ Check if the current platform is WSL
+ :return: True if the current platform is WSL
+ """
+ release_name = platform.uname().release
Review Comment:
Yeah I guess it hardly possible that macOS pass over this condition `if not has_wsl_interop and not microsoft_in_release and not wsl_conf:`
But who knows what the changes happen in the future conditions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org