You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "zach-overflow (via GitHub)" <gi...@apache.org> on 2026/09/14 08:19:11 UTC
[PR] Migrate Task SDK to `httpx2` [airflow]
zach-overflow opened a new pull request, #73116:
URL: https://github.com/apache/airflow/pull/73116
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
## What?
* Modifies `apache-airflow-task-sdk`'s dependency on `httpx` → `httpx2`. The Execution API `Client` is now an `httpx2.Client`.
* Adds `certifi` as an explicit `task-sdk` dependency — `client.py` imports it directly and it used to arrive via `httpx`, but `httpx2` depends on `truststore` instead. TLS behaviour is unchanged: the client always passes its own `ssl.SSLContext`.
* Moves `InProcessExecutionAPI.transport` / `.atransport` in `airflow-core` to `httpx2`, plus the call sites that build or drive them. Objects don't cross the `httpx`/`httpx2` boundary, and the only consumers of those transports are Task SDK clients. The rest of `airflow-core` stays on `httpx`.
* Teaches `shared/logging` to suppress and mute both stacks, since `airflow-core` and the Task SDK are now on different ones.
* Adds an [`ast-grep`](https://github.com/ast-grep/ast-grep) pre-commit check to prevent re-introducing `httpx` in the Task SDK, mirroring the providers one.
## Why?
* related: #70522
* follows: #72111
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes (please specify the tool below)
* Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
* Scope of AI use:
* I pointed Claude Code at the `task-sdk` `httpx` usage and it did the import replacements and dependency updates.
* It traced the cross-package boundary (in-process Execution API transports, `devel-common` test helpers, prek scripts) and migrated those call sites.
* It wrote the `ast-grep` rule and verified it fires.
* I reviewed the diff and wrote the PR description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "zach-overflow (via GitHub)" <gi...@apache.org>.
zach-overflow commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4122839343
##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -1336,18 +1336,19 @@ def __repr__(self):
return repr(self.detail)
-class ServerResponseError(httpx.HTTPStatusError):
- def __init__(self, message: str, *, request: httpx.Request, response: httpx.Response):
+class ServerResponseError(httpx2.HTTPStatusError):
Review Comment:
I had to do a bit of digging on this one, but my understanding is that `Client` and `ServerResponseError` are components of the Task sdk's _internals_. Neither of those classes are listed in the [public API reference](https://airflow.apache.org/docs/task-sdk/stable/api.html), and `airflow.sdk.api` is explicitly excluded from the generated docs. During normal task execution, the supervisor converts those `ServerResponseError` failures into `AirflowRuntimeError` in the task process before returning an error response.
The `devel-common` helpers do need to support both stacks because the compat jobs run against released SDK versions. That’s handled by conditionally selecting the right `httpx*` package which the installed `Client` subclasses.
I’d prefer to keep the SDK itself consistently on `httpx2` if possible. Is there a supported use case for directly constructing `Client` or catching its HTTP exceptions which I’m missing? If so, maybe we should account for that explicitly? A conditional import alone wouldn’t preserve existing `httpx` exception catches or make its transports compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala merged PR #73116:
URL: https://github.com/apache/airflow/pull/73116
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "zach-overflow (via GitHub)" <gi...@apache.org>.
zach-overflow commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4122846037
##########
devel-common/src/tests_common/test_utils/in_process_taskrun.py:
##########
@@ -55,27 +59,44 @@
_XCOM_PATH_PARTS = 5 # /xcoms/{dag_id}/{run_id}/{task_id}/{key}
+def resolve_sdk_httpx() -> ModuleType:
+ """Return the httpx implementation the *installed* Task SDK ``Client`` is built on.
+
+ Compat tests run this helper against released Airflow versions whose Task SDK is still
+ on ``httpx``, while the current one is on ``httpx2``. Transports and responses do not
+ cross that boundary — httpx asserts on its own stream types — so the transport MUST be
+ built from the same package the client subclasses.
+ """
Review Comment:
fixed in https://github.com/apache/airflow/pull/73116/commits/0c83ff214ec8dc6cccd3eb4a255bb652ca94fd24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4056553777
##########
devel-common/tests/unit/tests_common/test_utils/test_in_process_taskrun.py:
##########
@@ -0,0 +1,44 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+import httpx
+import httpx2
Review Comment:
`devel-common/pyproject.toml` declares neither? Works here because the venv installs multiple libraries where these dependencies are met. I feel like we should add `importorskip` with `httpx2`
##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -1336,18 +1336,19 @@ def __repr__(self):
return repr(self.detail)
-class ServerResponseError(httpx.HTTPStatusError):
- def __init__(self, message: str, *, request: httpx.Request, response: httpx.Response):
+class ServerResponseError(httpx2.HTTPStatusError):
Review Comment:
`ServerResponseError` is public API. Anyone catching `httpx.HTTPStatusError` or `httpx.RequestError` around an SDK call stops catching it, silently. Anyone passing `httpx.MockTransport` into `Client(transport=...)` gets a stream type failure too. We should support the compat here.
##########
airflow-core/src/airflow/api_fastapi/execution_api/app.py:
##########
@@ -422,9 +422,11 @@ async def always_allow(request: Request):
return self._app
+ # httpx2, not httpx: the only consumers are Task SDK clients, which subclass httpx2.Client,
+ # and transports do not cross the httpx/httpx2 package boundary.
Review Comment:
```suggestion
# httpx2, not httpx: the only consumers are Task SDK clients, which subclass httpx2.Client.
```
##########
devel-common/src/tests_common/test_utils/in_process_taskrun.py:
##########
@@ -55,27 +59,44 @@
_XCOM_PATH_PARTS = 5 # /xcoms/{dag_id}/{run_id}/{task_id}/{key}
+def resolve_sdk_httpx() -> ModuleType:
Review Comment:
Does this have to be public API? Usage is within the same module.
##########
devel-common/src/tests_common/test_utils/in_process_taskrun.py:
##########
@@ -55,27 +59,44 @@
_XCOM_PATH_PARTS = 5 # /xcoms/{dag_id}/{run_id}/{task_id}/{key}
+def resolve_sdk_httpx() -> ModuleType:
+ """Return the httpx implementation the *installed* Task SDK ``Client`` is built on.
+
+ Compat tests run this helper against released Airflow versions whose Task SDK is still
+ on ``httpx``, while the current one is on ``httpx2``. Transports and responses do not
+ cross that boundary — httpx asserts on its own stream types — so the transport MUST be
+ built from the same package the client subclasses.
+ """
Review Comment:
```suggestion
"""Return the httpx package the *installed* Task SDK ``Client`` subclasses.
Compat jobs pair this helper with a released SDK still on ``httpx``. A transport from the
wrong package trips httpx's own stream-type assertion.
"""
```
##########
shared/logging/src/airflow_shared/logging/structlog.py:
##########
@@ -420,22 +420,16 @@ def structlog_processors(
# structlog to ignore.
import contextlib
+ import importlib
import click
suppress: tuple[ModuleType, ...] = (click, contextlib)
- try:
- import httpcore
-
- suppress = (*suppress, httpcore)
- except ImportError:
- pass
- try:
- import httpx
-
- suppress = (*suppress, httpx)
- except ImportError:
- pass
+ # airflow-core is still on httpx while the Task SDK is on httpx2, so either (or both) may
+ # be installed; tracked at https://github.com/apache/airflow/issues/70522
+ for module_name in ("httpcore", "httpcore2", "httpx", "httpx2"):
Review Comment:
The new suppress entries and the new` "httpx2": {"level": "WARN"}` log-level entry have no test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4193609088
##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -1336,18 +1336,19 @@ def __repr__(self):
return repr(self.detail)
-class ServerResponseError(httpx.HTTPStatusError):
- def __init__(self, message: str, *, request: httpx.Request, response: httpx.Response):
+class ServerResponseError(httpx2.HTTPStatusError):
Review Comment:
Yeah, I think this is fine as you have it. As you say, this isn't really meant to be consumed directly (we probably should have called this `airflow.sdk._internal.api.client` in hindsight.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "zach-overflow (via GitHub)" <gi...@apache.org>.
zach-overflow commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4013455265
##########
airflow-core/newsfragments/73116.significant.rst:
##########
@@ -0,0 +1,20 @@
+The Task SDK now depends on ``httpx2`` instead of ``httpx``
Review Comment:
Sounds good - I wasn't sure if this would have end-user effects. Removed 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4005895545
##########
airflow-core/newsfragments/73116.significant.rst:
##########
@@ -0,0 +1,20 @@
+The Task SDK now depends on ``httpx2`` instead of ``httpx``
Review Comment:
This seems pretty internal - I am not sure if there are any user-side effects of that change (other than direct dependency - which is not really a user-facing change).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "zach-overflow (via GitHub)" <gi...@apache.org>.
zach-overflow commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4122844090
##########
devel-common/src/tests_common/test_utils/in_process_taskrun.py:
##########
@@ -55,27 +59,44 @@
_XCOM_PATH_PARTS = 5 # /xcoms/{dag_id}/{run_id}/{task_id}/{key}
+def resolve_sdk_httpx() -> ModuleType:
Review Comment:
fixed in https://github.com/apache/airflow/pull/73116/commits/0c83ff214ec8dc6cccd3eb4a255bb652ca94fd24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "zach-overflow (via GitHub)" <gi...@apache.org>.
zach-overflow commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4189386805
##########
task-sdk/pyproject.toml:
##########
@@ -53,8 +53,10 @@ dependencies = [
"asgiref>=3.11.1; python_version >= '3.14'",
"attrs>=24.2.0, !=25.2.0",
"babel>=2.17.0",
+ # Direct import in airflow.sdk.api.client; used to arrive via httpx, but httpx2 uses truststore.
+ "certifi>=2024.7.4",
Review Comment:
Good question, and I don't believe this was discussed in prior threads on this PR. Let me double check that we can do a simple swap with the `certifi` call(s) in `task-sdk/src/airflow/sdk/api/client.py` and update accordingly if possible. From the [official "migrating from httpx" docs](https://pydantic.dev/docs/httpx2/get-started/migration/#behavior-differences), they do mention this change, and I agree with their assessment that it should really only simplify things by using `truststore` over `ceritifi`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4193596338
##########
task-sdk/pyproject.toml:
##########
@@ -53,8 +53,10 @@ dependencies = [
"asgiref>=3.11.1; python_version >= '3.14'",
"attrs>=24.2.0, !=25.2.0",
"babel>=2.17.0",
+ # Direct import in airflow.sdk.api.client; used to arrive via httpx, but httpx2 uses truststore.
+ "certifi>=2024.7.4",
Review Comment:
Lovely.
I imagine this would help people have corporate MITM ssl proxies too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4189215480
##########
task-sdk/pyproject.toml:
##########
@@ -53,8 +53,10 @@ dependencies = [
"asgiref>=3.11.1; python_version >= '3.14'",
"attrs>=24.2.0, !=25.2.0",
"babel>=2.17.0",
+ # Direct import in airflow.sdk.api.client; used to arrive via httpx, but httpx2 uses truststore.
+ "certifi>=2024.7.4",
Review Comment:
Sorry if this was discussed already (the perils of resolving conversations) why depend on "certifi" -- we don't _strictly_ need certifi do we (and "the code uses it today" isn't a reason to keep it), and if httpx2 uses something different, is it worth us having two cert trust stores? Why not switch to truststore too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "zach-overflow (via GitHub)" <gi...@apache.org>.
zach-overflow commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4189535853
##########
task-sdk/pyproject.toml:
##########
@@ -53,8 +53,10 @@ dependencies = [
"asgiref>=3.11.1; python_version >= '3.14'",
"attrs>=24.2.0, !=25.2.0",
"babel>=2.17.0",
+ # Direct import in airflow.sdk.api.client; used to arrive via httpx, but httpx2 uses truststore.
+ "certifi>=2024.7.4",
Review Comment:
@ashb I'm in agreement. Addressed in https://github.com/apache/airflow/pull/73116/commits/954e4451659f7662776ec019de6b4685db6d540c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #73116:
URL: https://github.com/apache/airflow/pull/73116#issuecomment-6223569094
### Backport successfully created: v3-4-test
Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>✅</td>
<td>v3-4-test</td>
<td><a href="https://github.com/apache/airflow/pull/74891"><img src="https://img.shields.io/badge/PR-74891-blue" alt="PR Link"></a></td>
</tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #73116:
URL: https://github.com/apache/airflow/pull/73116#issuecomment-5802389195
<!-- uv-lock-rebase-notice -->
`uv.lock` on `main` just moved via [#70960](https://redirect.github.com/apache/airflow/pull/70960) ("Add SRV DNS record support to HTTP Operator"), commit [`b6acedb`](https://redirect.github.com/apache/airflow/commit/b6acedbab23f9a3bbfb307b5381ac2ffbcfc4255) and this PR currently conflicts.
Quickest fix:
```bash
git fetch upstream main && git rebase upstream/main
rm uv.lock && uv lock
git add uv.lock && git rebase --continue
git push --force-with-lease
```
_Automated nudge — ignore if you're not ready to rebase. This comment is updated in place on future `uv.lock` bumps._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4193632375
##########
devel-common/src/tests_common/test_utils/in_process_taskrun.py:
##########
@@ -55,27 +59,42 @@
_XCOM_PATH_PARTS = 5 # /xcoms/{dag_id}/{run_id}/{task_id}/{key}
+def _resolve_sdk_httpx() -> ModuleType:
+ """Return the httpx package the *installed* Task SDK ``Client`` subclasses.
+
+ Compat jobs pair this helper with a released SDK still on ``httpx``. A transport from the
Review Comment:
```suggestion
Needed as these test_utils run against multiple versions of Airflow/sdk. This can be
removed when the oldest version we test against is Airflow 3.4.
Compat jobs pair this helper with a released SDK still on ``httpx``. A transport from the
```
##########
task-sdk/.pre-commit-config.yaml:
##########
@@ -65,3 +65,12 @@ repos:
pass_filenames: false
files: ^.*\.py$
require_serial: true
+ - repo: https://github.com/boidolr/ast-grep-pre-commit
+ rev: 7bae9c67e1a003cd5ef629c64f0466dc46556b3b # frozen: 0.45.3
+ hooks:
+ - id: ast-grep
Review Comment:
I don't think we need to introduce ast-grep and depend on a new providers/module for this. Ruff has banned imports I think.
Please see if you can do it via task-sdk/pyproject.toml?
##########
task-sdk/tests/task_sdk/api/test_client.py:
##########
@@ -1941,29 +1936,53 @@ def handle_request(request: httpx.Request) -> httpx.Response:
assert result.responded_at == timezone.datetime(2025, 7, 3, 0, 0, 0)
-class TestSSLContextCaching:
+class TestSSLContext:
Review Comment:
Needless rename?
##########
task-sdk/tests/task_sdk/api/test_client.py:
##########
@@ -20,15 +20,19 @@
import json
import pickle
import sys
-from datetime import datetime, timezone as dt_timezone
+from datetime import datetime, timedelta, timezone as dt_timezone
from typing import TYPE_CHECKING
from unittest import mock
-import certifi
-import httpx
+import httpx2
import pytest
import time_machine
+import truststore
import uuid6
+from cryptography import x509
+from cryptography.hazmat.primitives import hashes, serialization
+from cryptography.hazmat.primitives.asymmetric import ec
+from cryptography.x509.oid import NameOID
Review Comment:
These are scary imports :) Can we scope these to just the one `ca_file` that needs them? It would make me feel happier that the "unsafe" bit is limited to one area.
##########
task-sdk/tests/task_sdk/api/test_client.py:
##########
@@ -1941,29 +1936,53 @@ def handle_request(request: httpx.Request) -> httpx.Response:
assert result.responded_at == timezone.datetime(2025, 7, 3, 0, 0, 0)
-class TestSSLContextCaching:
+class TestSSLContext:
@pytest.fixture(autouse=True)
def clear_ssl_context_cache(self):
Client._get_ssl_context_cached.cache_clear()
yield
Client._get_ssl_context_cached.cache_clear()
- def test_cache_hit_on_same_parameters(self):
- ca_file = certifi.where()
+ @pytest.fixture
+ def ca_file(self, tmp_path):
+ key = ec.generate_private_key(ec.SECP256R1())
+ name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, "test-ca")])
+ now = datetime.now(dt_timezone.utc)
+ cert = (
+ x509.CertificateBuilder()
+ .subject_name(name)
+ .issuer_name(name)
+ .public_key(key.public_key())
+ .serial_number(x509.random_serial_number())
+ .not_valid_before(now)
+ .not_valid_after(now + timedelta(days=1))
+ .add_extension(x509.BasicConstraints(ca=True, path_length=None), critical=True)
+ .sign(key, hashes.SHA256())
+ )
+ path = tmp_path / "ca.pem"
+ path.write_bytes(cert.public_bytes(serialization.Encoding.PEM))
+ return str(path)
+
+ @mock.patch.object(truststore.SSLContext, "load_verify_locations", autospec=True)
Review Comment:
I'm slightly nervous about mocking this.
Also: if we are mocking this, do we actually need a real CA file on disk? If feels like it should be one or the other, but not both?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Migrate Task SDK to `httpx2` [airflow]
Posted by "zach-overflow (via GitHub)" <gi...@apache.org>.
zach-overflow commented on code in PR #73116:
URL: https://github.com/apache/airflow/pull/73116#discussion_r4123132182
##########
shared/logging/src/airflow_shared/logging/structlog.py:
##########
@@ -420,22 +420,16 @@ def structlog_processors(
# structlog to ignore.
import contextlib
+ import importlib
import click
suppress: tuple[ModuleType, ...] = (click, contextlib)
- try:
- import httpcore
-
- suppress = (*suppress, httpcore)
- except ImportError:
- pass
- try:
- import httpx
-
- suppress = (*suppress, httpx)
- except ImportError:
- pass
+ # airflow-core is still on httpx while the Task SDK is on httpx2, so either (or both) may
+ # be installed; tracked at https://github.com/apache/airflow/issues/70522
+ for module_name in ("httpcore", "httpcore2", "httpx", "httpx2"):
Review Comment:
added 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org