You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "dheerajturaga (via GitHub)" <gi...@apache.org> on 2025/11/22 18:47:54 UTC
[PR] Prevent duplicate edge workers unless existing worker is offline or unkown [airflow]
dheerajturaga opened a new pull request, #58586:
URL: https://github.com/apache/airflow/pull/58586
Add validation to the edge worker registration endpoint to prevent
launching multiple workers with the same hostname. If a worker with
the same name already exists in an active state (running, idle,
starting, terminating, or maintenance), the registration will fail
with HTTP 409 CONFLICT. Workers can only reuse a name if the existing
worker is in OFFLINE, UNKNOWN, or OFFLINE_MAINTENANCE state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Prevent duplicate edge workers unless existing worker is offline or unkown [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl merged PR #58586:
URL: https://github.com/apache/airflow/pull/58586
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Prevent duplicate edge workers unless existing worker is offline or unkown [airflow]
Posted by "dheerajturaga (via GitHub)" <gi...@apache.org>.
dheerajturaga commented on PR #58586:
URL: https://github.com/apache/airflow/pull/58586#issuecomment-3568229115
> Can you explicitly handle the exception on the client and generate a better log message in the console? e.g. like we have for version conflicts in providers/edge3/src/airflow/providers/edge3/cli/api_client.py:133
Does this look good to you? I made this update
```
2025-11-23T18:30:05.778730Z [info ] Starting worker with API endpoint http://localhost:8080/edge_worker/v1/rpcapi [airflow.providers.edge3.cli.edge_command] loc=edge_command.py:80
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
____ __ _ __ __
/ __/__/ /__ ____ | | /| / /__ ____/ /_____ ____
/ _// _ / _ `/ -_) | |/ |/ / _ \/ __/ '_/ -_) __/
/___/\_,_/\_, /\__/ |__/|__/\___/_/ /_/\_\\__/_/
/___/
2025-11-23T18:30:05.924089Z [error ] A worker with the name 'breeze' is already active. Please ensure worker names are unique, or stop the existing worker before starting a new one. [airflow.providers.edge3.cli.worker] loc=worker.py:270
A worker with the name 'breeze' is already active. Please ensure worker names are unique, or stop the existing worker before starting a new one.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Prevent duplicate edge workers unless existing worker is offline or unkown [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #58586:
URL: https://github.com/apache/airflow/pull/58586#issuecomment-3567721653
Just realized: This PR will now lower a bit comfort though, when using breeze and then `stop_airflow` the docker is killed and if you then re-start the previous worker state is not cleaned and restarting directly with breeze will raise the HTTP 409 then :-) But still OK in my view.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Prevent duplicate edge workers unless existing worker is offline or unkown [airflow]
Posted by "dheerajturaga (via GitHub)" <gi...@apache.org>.
dheerajturaga commented on PR #58586:
URL: https://github.com/apache/airflow/pull/58586#issuecomment-3568231574
> Just realized: This PR will now lower a bit comfort though, when using breeze and then `stop_airflow` the docker is killed and if you then re-start the previous worker state is not cleaned and restarting directly with breeze will raise the HTTP 409 then :-) But still OK in my view.
Thats true, looks like the state does not get cleared. You would need to wait a few minutes for the api server to determine worker hearbeat is missing and change to unknown state, then you will be able to launch. OR just launch with `-d` (which I generally always use when testing).
I don't think this is a big deal though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org