You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jixianyihao (via GitHub)" <gi...@apache.org> on 2025/06/20 12:33:27 UTC
[I] worker receive msg [airflow]
jixianyihao opened a new issue, #51962:
URL: https://github.com/apache/airflow/issues/51962
### Apache Airflow version
3.0.2
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
two different worker receive the same task
### What you think should happen instead?
_No response_
### How to reproduce
3.0.2 helm chart deploy， multi scheduler 7， worker： 20， api：3
### Operating System
k8s
### Versions of Apache Airflow Providers
helm chart
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
2025-06-20 19:42:21.306
[2025-06-20T19:42:21.305+0800] {scheduler_job_runner.py:842} INFO - Setting external_id for <TaskInstance: dsds.task_run_application scheduled__2025-06-10T04:05:00+00:00 [queued]> to f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71
2025-06-20 19:45:15.518
2025-06-20 11:45:15.518051 [info ] Task execute_workload[f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] received [celery.worker.strategy]
2025-06-20 19:45:15.528
2025-06-20 11:45:15.527881 [info ] [f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] Executing workload in Celery: token='eyJ***' ti=TaskInstance(id=UUID('01978d23-23fc-72fa-9f99-bdd352265f31'), task_id='task_run_application', dag_id='x', run_id='scheduled__2025-06-10T04:05:00+00:00', try_number=1, map_index=-1, pool_slots=1, queue='default', priority_weight=1, executor_config=None, parent_context_carrier={}, context_carrier={}, queued_dttm=None) dag_rel_path=PurePosixPath('flink/airflow_task/dsds/dsds.py') bundle_info=BundleInfo(name='dags-folder', version=None) log_path='dag_id=dsds/run_id=scheduled__2025-06-10T04:05:00+00:00/task_id=task_run_application/attempt=1.log' type='ExecuteTask' [airflow.providers.celery.executors.celery_executor_utils]
2025-06-20 19:45:25.360
2025-06-20 11:45:25.359910 [info ] Task execute_workload[f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] received [celery.worker.strategy]
2025-06-20 19:45:25.369
2025-06-20 11:45:25.368977 [info ] [f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] Executing workload in Celery: token='eyJ***' ti=TaskInstance(id=UUID('01978d23-23fc-72fa-9f99-bdd352265f31'), task_id='task_run_application', dag_id='dsds', run_id='scheduled__2025-06-10T04:05:00+00:00', try_number=1, map_index=-1, pool_slots=1, queue='default', priority_weight=1, executor_config=None, parent_context_carrier={}, context_carrier={}, queued_dttm=None) dag_rel_path=PurePosixPath('flink/airflow_task/dsds/dsds.py') bundle_info=BundleInfo(name='dags-folder', version=None) log_path='dag_id=dsds/run_id=scheduled__2025-06-10T04:05:00+00:00/task_id=task_run_application/attempt=1.log' type='ExecuteTask' [airflow.providers.celery.executors.celery_executor_utils]
2025-06-20 19:45:26.848
2025-06-20 11:45:25.492400 [error ] Task execute_workload[f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] raised unexpected: ServerResponseError('Server returned error') [celery.app.trace]
2025-06-20 19:45:26.849
│ │ 'id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', │ │
2025-06-20 19:45:26.849
│ │ root_id = 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71' │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'shadow': None, │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'parent_id': None, │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'shadow': None, │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'parent_id': None, │ │
2025-06-20 19:45:26.849
│ │ 'correlation_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', │ │
2025-06-20 19:45:26.849
│ │ 'correlation_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', │ │
2025-06-20 19:45:26.849
│ │ uuid = 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71' │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'shadow': None, 'eta': None, 'expires': │ │
2025-06-20 19:45:26.849
│ │ 'root_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'parent_id': None, 'argsrepr': │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'shadow': None, 'eta': None, 'expires': │ │
2025-06-20 19:45:26.849
│ │ 'root_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'parent_id': None, 'argsrepr': │ │
2025-06-20 19:45:26.849
│ │ 'delivery_mode': 2, 'correlation_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'hostname': │ │
2025-06-20 19:45:26.849
│ │ celery_task_id = 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71' │ │
2025-06-20 19:45:28.453
2025-06-20 11:45:28.452977 [info ] Task execute_workload[f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] succeeded in 12.933839422650635s: None [celery.app.trace]
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] worker receive msg [airflow]
Posted by "jixianyihao (via GitHub)" <gi...@apache.org>.
jixianyihao commented on issue #51962:
URL: https://github.com/apache/airflow/issues/51962#issuecomment-2994263905
is there any bug? two different worker receive the same task
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] worker receive msg [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #51962:
URL: https://github.com/apache/airflow/issues/51962#issuecomment-2991403426
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] worker receive msg [airflow]
Posted by "jixianyihao (via GitHub)" <gi...@apache.org>.
jixianyihao commented on issue #51962:
URL: https://github.com/apache/airflow/issues/51962#issuecomment-2994263535
any one can analyze this, we use it in production, now we rollback to 2.10.5, it is ok for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] worker receive msg [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #51962: worker receive msg
URL: https://github.com/apache/airflow/issues/51962
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] worker receive msg [airflow]
Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on issue #51962:
URL: https://github.com/apache/airflow/issues/51962#issuecomment-3510977913
Hi @jixianyihao could you please test on this latest Airflow version 3.1.2, and let us know if the problem persists?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] worker receive msg [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #51962:
URL: https://github.com/apache/airflow/issues/51962#issuecomment-4464771513
This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] worker receive msg [airflow]
Posted by "vikramkoka (via GitHub)" <gi...@apache.org>.
vikramkoka commented on issue #51962:
URL: https://github.com/apache/airflow/issues/51962#issuecomment-3192946577
@jixianyihao Thank you for reporting this.
Can you please also share the database (and version) information?
Also, you had indicated that you would be willing to submit a PR. So can you please share a draft PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] worker receive msg [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #51962:
URL: https://github.com/apache/airflow/issues/51962#issuecomment-4523572455
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org