You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/12/06 23:07:15 UTC
[GitHub] [airflow] philipherrmann opened a new issue #20088: httpx breaking API change for redirect parameter causes crash
philipherrmann opened a new issue #20088:
URL: https://github.com/apache/airflow/issues/20088
### Apache Airflow Provider(s)
google
### Versions of Apache Airflow Providers
apache-airflow-providers-google==6.1.0
### Apache Airflow version
2.2.2 (latest released)
### Operating System
Debian GNU/Linux 10 (buster)
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### What happened
When using the CloudSQLExecuteQueryOperator the Cloud SQL Proxy Download fails due to a breaking API change in the httpx dependency, logging the following traceback:
```
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/google/cloud/operators/cloud_sql.py", line 1056, in _execute_query
cloud_sql_proxy_runner.start_proxy()
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/google/cloud/hooks/cloud_sql.py", line 558, in start_proxy
self._download_sql_proxy_if_needed()
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/google/cloud/hooks/cloud_sql.py", line 502, in _download_sql_proxy_if_needed
response = httpx.get(download_url, allow_redirects=True)
TypeError: get() got an unexpected keyword argument 'allow_redirects'
```
The keyword allow_redirects has been renamed to follow_redirects in this PR https://github.com/encode/httpx/pull/1808. One should probably fix either of those options and pin a suitable version of httpx.
### What you expected to happen
_No response_
### How to reproduce
_No response_
### Anything else
task = CloudSQLExecuteQueryOperator(
task_id="test",
gcp_cloudsql_conn_id="postgres-google-cloud-sql-connection",
trigger_rule=TriggerRule.ALL_SUCCESS,
sql="""
SELECT %s
""",
parameters=["test"],
)
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #20088:
URL: https://github.com/apache/airflow/issues/20088
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-993481614
> If you change the parameter name, you must change the constraint. Otherwise sure, give it a go.
It's OK what you did - changing setup.* is the right way to go. Constraints will update automatically
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-987614616
Note that when you fix this, you’ll need to also change `httpx` in `setup.cfg` to `httpx>=0.20`. (Or if that’s not viable due to maybe providers need the old argument name, pin `setup.cfg` to `httpx<0.20` for now and _don’t_ change the argument.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-987452522
@philipherrmann Assigned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-1004249734
fixed by https://github.com/apache/airflow/pull/20239
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] philipherrmann commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
philipherrmann commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-988672596
So the quick fix is in place for the current version (2.2.2) of airflow. Would you still like me to create a PR for use in future versions of airflow, which
1. changes the parameter name to `follow_redirects` in httpx calls and
2. maybe even changes the dependency constraint to `httpx~=0.21.1`?
No worries if you'd prefer to do that yourself, I have a bunch things to do :) Otherwise, I would give it a try next weekend probably.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] philipherrmann commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
philipherrmann commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-993324511
`If you change the parameter name, you must change the constraint. Otherwise sure, give it a go.`
Surely, *someone* must change the constraints. I meant to offer that, but as they are in this extra branches I am slightly confused. Impossible to get those two contributions into the same PR, isn't it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-988698465
If you change the parameter name, you _must_ change the constraint. Otherwise sure, give it a go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-987820434
This is what one of the nice properties of the constraint mechanism of ours, that we can fix such problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-987828752
Ok. It was only for 2.2.2, The 2.2.1 version already had httpx == 0.19.0. I fixed the constraints now, so anyone installing airfllow in the "constraint" way will not have the problem. I am also building and pushing the images for 2.2.2 to include that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-987828752
Ok. It was only for 2.2.2, The 2.2.1 version already had httpx == 0.19.0. I fixed the constraints now, so anyone installing airlflow in the "constraint" way will not have the problem. I am also building and pushing the images for 2.2.2 to include that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-987338779
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20088: httpx breaking API change for redirect parameter causes crash
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20088:
URL: https://github.com/apache/airflow/issues/20088#issuecomment-987819675
Really "nice" from httpx POV to make this breaking change here :(. httpx >=0.20 in this case is a good idea.
I will also manually update constraints to 0.19 for the past releases of airflow that had 0.20.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org