You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Adityakk9031 (via GitHub)" <gi...@apache.org> on 2026/08/02 17:52:48 UTC
[PR] Fix Windows compatibility in prek scripts and POSIX-only test skip gu… [airflow]
Adityakk9031 opened a new pull request, #70946:
URL: https://github.com/apache/airflow/pull/70946
Close: #69464 (follow-up)
Three Windows-compatibility fixes split out from #69504 per maintainer review:
common_prek_utils.py: explicit encoding="utf-8" on all read_text() calls prevents UnicodeDecodeError on Windows where the default codec is cp1252
shared/observability/.../stats.py: guard os.register_at_fork with hasattr since it is POSIX-only
test_supervisor.py / test_task_runner.py: skip on Windows with pytestmark rather than stubbing out POSIX syscalls — CI on Linux runs all tests as normal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Windows compatibility in prek scripts and POSIX-only test skip gu… [airflow]
Posted by "Adityakk9031 (via GitHub)" <gi...@apache.org>.
Adityakk9031 commented on PR #70946:
URL: https://github.com/apache/airflow/pull/70946#issuecomment-5159615472
@potiuk have a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Windows compatibility in prek scripts and guard POSIX-only calls [airflow]
Posted by "Adityakk9031 (via GitHub)" <gi...@apache.org>.
Adityakk9031 commented on code in PR #70946:
URL: https://github.com/apache/airflow/pull/70946#discussion_r4096628457
##########
scripts/ci/prek/common_prek_utils.py:
##########
@@ -118,7 +118,7 @@ def run_command(*args, **kwargs) -> None:
def read_airflow_version() -> str:
- ast_obj = ast.parse((AIRFLOW_CORE_SOURCES_PATH / "airflow" / "__init__.py").read_text())
+ ast_obj = ast.parse((AIRFLOW_CORE_SOURCES_PATH / "airflow" / "__init__.py").read_text(encoding="utf-8"))
Review Comment:
On Windows, Python's default encoding for open() and Path.read_text() is still based on the system locale (locale.getpreferredencoding(False), typically cp1252 on Western Windows locales), not UTF-8.
Python only makes UTF-8 the default across all platforms starting in Python 3.15 (PEP 686). On Python 3.9–3.14 without UTF-8 mode explicitly enabled, reading files with non-ASCII characters without encoding=\utf-8\ raises UnicodeDecodeError: 'charmap' codec can't decode byte 0x81.... Explicitly passing encoding=\utf-8\ ensures consistent cross-platform behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Task SDK: Skip POSIX-only supervisor and task runner tests on Windows [airflow]
Posted by "Adityakk9031 (via GitHub)" <gi...@apache.org>.
Adityakk9031 commented on PR #70946:
URL: https://github.com/apache/airflow/pull/70946#issuecomment-5819384676
@potiuk @uros-b Per feedback, I have split the changes into focused micro-PRs by area:
- **Prek / Dev tools**: #73679
- **Observability / Shared**: #73680
- **Task SDK Tests (This PR)**: Scoped strictly to skipping POSIX-only supervisor and task runner tests on Windows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Task SDK: Skip POSIX-only supervisor and task runner tests on Windows [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on PR #70946:
URL: https://github.com/apache/airflow/pull/70946#issuecomment-6289921883
@Adityakk9031 I don't think we say anywhere that Airflow support Windows!
If your goal is to ensure it runs with Windows, please upload a demo window or screenshot of it working end to end
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Windows compatibility in prek scripts and guard POSIX-only calls [airflow]
Posted by "Adityakk9031 (via GitHub)" <gi...@apache.org>.
Adityakk9031 commented on PR #70946:
URL: https://github.com/apache/airflow/pull/70946#issuecomment-5819126629
@potiuk @uros-b The PR title has been updated as requested.
Regarding the scope: the entire PR is only +16 / -6 lines across 4 files to resolve basic Windows development/testing blockers:
- \scripts/ci/prek/common_prek_utils.py\: explicit UTF-8 encoding for \ ead_text\ (replied to @ashb's question on the review thread as well).
- \shared/observability/.../stats.py\: guard \os.register_at_fork\ with \hasattr\ for Windows compatibility.
- \ est_supervisor.py\ / \ est_task_runner.py\: POSIX-only test skip markers on Windows.
All CI checks are passing. Could we keep this unified since the diff is very small and cohesive around Windows compatibility, or would you still prefer it split into micro-PRs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Windows compatibility in prek scripts and guard POSIX-only calls [airflow]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #70946:
URL: https://github.com/apache/airflow/pull/70946#issuecomment-5809048799
Thank you @Adityakk9031, please ping @potiuk here for further review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Task SDK: Skip POSIX-only supervisor and task runner tests on Windows [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #70946:
URL: https://github.com/apache/airflow/pull/70946#discussion_r4272341385
##########
scripts/ci/prek/common_prek_utils.py:
##########
@@ -118,7 +118,7 @@ def run_command(*args, **kwargs) -> None:
def read_airflow_version() -> str:
- ast_obj = ast.parse((AIRFLOW_CORE_SOURCES_PATH / "airflow" / "__init__.py").read_text())
+ ast_obj = ast.parse((AIRFLOW_CORE_SOURCES_PATH / "airflow" / "__init__.py").read_text(encoding="utf-8"))
Review Comment:
It was really only 3.15? Huh, I thought it was ages ago. I am maybe confusing python source encoding and "opening a file for reading"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Windows compatibility in prek scripts and guard POSIX-only calls [airflow]
Posted by "Adityakk9031 (via GitHub)" <gi...@apache.org>.
Adityakk9031 commented on PR #70946:
URL: https://github.com/apache/airflow/pull/70946#issuecomment-5808524223
@uros-b changed please merge it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Windows compatibility in prek scripts and POSIX-only test skip gu… [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #70946:
URL: https://github.com/apache/airflow/pull/70946#issuecomment-5805212052
This pull request has been automatically marked as stale because the author has not responded to a request for more information. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Windows compatibility in prek scripts and POSIX-only test skip gu… [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #70946:
URL: https://github.com/apache/airflow/pull/70946#issuecomment-5277957950
Please split those even further - each for different area. Some of them some prek, some tests, some production.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Windows compatibility in prek scripts and POSIX-only test skip gu… [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #70946:
URL: https://github.com/apache/airflow/pull/70946#discussion_r3820643464
##########
scripts/ci/prek/common_prek_utils.py:
##########
@@ -118,7 +118,7 @@ def run_command(*args, **kwargs) -> None:
def read_airflow_version() -> str:
- ast_obj = ast.parse((AIRFLOW_CORE_SOURCES_PATH / "airflow" / "__init__.py").read_text())
+ ast_obj = ast.parse((AIRFLOW_CORE_SOURCES_PATH / "airflow" / "__init__.py").read_text(encoding="utf-8"))
Review Comment:
Even on windows I though the default encoding for python opening files is UTF-8?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org