You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "adrianrego (via GitHub)" <gi...@apache.org> on 2024/02/23 19:25:20 UTC
[I] HttpHookAsync does not support endpoint as a full url. Inconsistent behavior compared to HttpHook. [airflow]
adrianrego opened a new issue, #37664:
URL: https://github.com/apache/airflow/issues/37664
### Apache Airflow version
2.8.1
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
When creating a deferrable HTTPSensor with no connection and full url for endpoint an invalid url is constructed:
```
HttpSensor(endpoint="https://someurl.com", http_conn_id="", deferrable=True)
```
URL poked via HttpHookAsync is: `/http://someurl.com`
This is due to the assumption that base_url is passed into the hook or pulled from an http connection: https://github.com/apache/airflow/blob/main/airflow/providers/http/hooks/http.py#L362
This is not an issue with HttpHook as it handles the base where base_url is None or endpoint is a complete url.
https://github.com/apache/airflow/blob/main/airflow/providers/http/hooks/http.py#L264
Traceback:
```
[2024-02-22 19:19:47,225] Trigger failed:
Traceback (most recent call last):
File "/opt/conda/lib/python3.11/site-packages/airflow/jobs/triggerer_job_runner.py", line 529, in cleanup_finished_triggers
result = details["task"].result()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/airflow/jobs/triggerer_job_runner.py", line 601, in run_trigger
async for event in trigger.run():
File "/opt/conda/lib/python3.11/site-packages/airflow/providers/http/triggers/http.py", line 183, in run
await hook.run(
File "/opt/conda/lib/python3.11/site-packages/airflow/providers/http/hooks/http.py", line 383, in run
response = await request_func(
^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/aiohttp/client.py", line 545, in _request
req = self._request_class(
^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 326, in __init__
self.update_host(url)
File "/opt/conda/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 400, in update_host
raise InvalidURL(url)
aiohttp.client_exceptions.InvalidURL: /https://ftpprd.ncep.noaa.gov/data/nccf/com/hrrr/prod/hrrr.20240222/conus/hrrr.t18z.wrfsfcf25.grib2
```
### What you think should happen instead?
HttpHookAsync should construct the url in a similar fashion to HttpHook and support a complete url via the `endpoint` kwarg. The original Astronomer HttpHookAsync also had similar behavior:
https://github.com/astronomer/astronomer-providers/blob/1.18.4/astronomer/providers/http/hooks/http.py#L103
### How to reproduce
```
HttpSensor(endpoint="https://someurl.com", http_conn_id="", deferrable=True)
```
### Operating System
All
### Versions of Apache Airflow Providers
apache-airflow-providers-http==4.9.1
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] HttpHookAsync does not support endpoint as a full url. Inconsistent behavior compared to HttpHook. [airflow]
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis closed issue #37664: HttpHookAsync does not support endpoint as a full url. Inconsistent behavior compared to HttpHook.
URL: https://github.com/apache/airflow/issues/37664
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] HttpHookAsync does not support endpoint as a full url. Inconsistent behavior compared to HttpHook. [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #37664:
URL: https://github.com/apache/airflow/issues/37664#issuecomment-1961870793
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org