You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kaxil (via GitHub)" <gi...@apache.org> on 2025/04/24 19:09:59 UTC
[PR] Make default execution server URL be relative to API Base URL [airflow]
kaxil opened a new pull request, #49747:
URL: https://github.com/apache/airflow/pull/49747
The execution API server URL is now more flexible and configurable:
- Remove hardcoded default URL "http://localhost:8080/execution/"
- Add fallback mechanism to construct URL from base_url config
- Default is now `"{BASE_URL}/execution/"` where BASE_URL comes from `api.base_url` config
related: https://github.com/apache/airflow/issues/49725
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #49747:
URL: https://github.com/apache/airflow/pull/49747#discussion_r2059105490
##########
airflow-core/src/airflow/executors/local_executor.py:
##########
@@ -108,6 +108,10 @@ def _execute_work(log: logging.Logger, workload: workloads.ExecuteTask) -> None:
from airflow.sdk.execution_time.supervisor import supervise
setproctitle(f"airflow worker -- LocalExecutor: {workload.ti.id}")
+
+ base_url = conf.get("api", "base_url", fallback="/")
+ default_execution_api_server = f"{base_url.rstrip('/')}/execution/"
Review Comment:
The problem is that they are with separate providers. So keeping the logic in central location has a big disadvantage where that provider won't work until that version is released!
So the repetition is intentional
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #49747:
URL: https://github.com/apache/airflow/pull/49747#issuecomment-2874320501
Yes new provider wave is coming - but if you want you can easily (now) build provider on your own from main and test it - or even install it via github URL:
```
pip install "apache-airflow-providers-celery @ git+https://github.com/apache/airflow/#subdirectory=providers/celery"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #49747:
URL: https://github.com/apache/airflow/pull/49747#discussion_r2059100670
##########
airflow-core/src/airflow/executors/local_executor.py:
##########
@@ -108,6 +108,10 @@ def _execute_work(log: logging.Logger, workload: workloads.ExecuteTask) -> None:
from airflow.sdk.execution_time.supervisor import supervise
setproctitle(f"airflow worker -- LocalExecutor: {workload.ti.id}")
+
+ base_url = conf.get("api", "base_url", fallback="/")
+ default_execution_api_server = f"{base_url.rstrip('/')}/execution/"
Review Comment:
Can we just add a property or extract this onto the base executor or somewhere relevant?
Too many repetitions. It will be easy to miss in case of a refactor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil merged PR #49747:
URL: https://github.com/apache/airflow/pull/49747
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #49747:
URL: https://github.com/apache/airflow/pull/49747#issuecomment-2875788408
> This journey from Airflow 2 to 3 has made me reluctant to be the first to "eat the crab" again. And I hate being proven right once more… after building and installing the latest Celery provider from main, I'm still getting bombarded with ConnectError: [Errno 111] Connection refused if I don't set the AIRFLOW__CORE__EXECUTION_API_SERVER_URL environment variable somewhere.
However, we greatly appreciate your efforts @zachliu -> yes, there are teething issues, and being an open-source project, it's almost a given that the first brave users who will start testing and using Airflow 3 will find things that we have not thoguht about and that's the only way we can actually get to know about those issues and fix them. There is no software with bugs, and no amount of "lab" tesitng is able to find all those - especially if you give your software for free and there is noone who will pay for that extra testing. So effectively this is the price you pay for using sotware that cost you `0 USD`. And we consider it a great contribution from people like you who are brave enough and supportive enough and understand that there might be issues - what we offer instead is a close cooperation, trying to find mitigations and offer ways to try and tests main versions and RC candidates as soon as we have them.
I think - to be honest - it was great getting your initial feedback and issue reports and we will continue to diagnose and fix issues as they come. I think the power of Airlfow come from the power of the community that is both helpful and understands that there are some initial teething pains - and goes through those pains together with us.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "zachliu (via GitHub)" <gi...@apache.org>.
zachliu commented on PR #49747:
URL: https://github.com/apache/airflow/pull/49747#issuecomment-2877125602
@potiuk Thank you for the encouragement, Professor—I was just venting. I’ll probably never stop being the first to eat the crab :joy:
As for the matter at hand, the issue seems to stem from a subtle difference between `localhost` and `<hostname>`. When specifying the execution API URL, it likes the `<hostname>`—such as the service name `apiserver` or `webserver`—like so:
`http://webserver:<port>/execution`. Should be fine in a production environment but not for local runs when my `base_url = http://localhost:<port>/`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #49747:
URL: https://github.com/apache/airflow/pull/49747#discussion_r2059108875
##########
airflow-core/src/airflow/executors/local_executor.py:
##########
@@ -108,6 +108,10 @@ def _execute_work(log: logging.Logger, workload: workloads.ExecuteTask) -> None:
from airflow.sdk.execution_time.supervisor import supervise
setproctitle(f"airflow worker -- LocalExecutor: {workload.ti.id}")
+
+ base_url = conf.get("api", "base_url", fallback="/")
+ default_execution_api_server = f"{base_url.rstrip('/')}/execution/"
Review Comment:
Valid point, seems fine. Yeah we will not be able to do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "zachliu (via GitHub)" <gi...@apache.org>.
zachliu commented on PR #49747:
URL: https://github.com/apache/airflow/pull/49747#issuecomment-2874277614
> as that provider won't work until that Airflow version is released!
should we expect new releases of the providers in a few days? for example the latest `apache-airflow-providers-celery==3.10.6
` doesn't have the fix yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Make default execution server URL be relative to API Base URL [airflow]
Posted by "zachliu (via GitHub)" <gi...@apache.org>.
zachliu commented on PR #49747:
URL: https://github.com/apache/airflow/pull/49747#issuecomment-2874825490
> Yes new provider wave is coming - but if you want you can easily (now) build provider on your own from main and test it - or even install it via github URL:
>
> ```
> pip install "apache-airflow-providers-celery @ git+https://github.com/apache/airflow/#subdirectory=providers/celery"
> ```
This journey from Airflow 2 to 3 has made me reluctant to be the first to "eat the crab" again. And I hate being proven right once more… after building and installing the latest Celery provider from main, I'm still getting bombarded with `ConnectError: [Errno 111] Connection refused` if I don't set the `AIRFLOW__CORE__EXECUTION_API_SERVER_URL` environment variable somewhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org