You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "bangbangDong (via GitHub)" <gi...@apache.org> on 2024/10/24 19:17:00 UTC
[D] An error occurred in webservice， request： /task_stats [airflow]
GitHub user bangbangDong created a discussion: An error occurred in webservice， request： /task_stats
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.10.0
### What happened?
10.1.4.76 - - [23/Oct/2024:16:39:41 +0800] "POST /dag_stats HTTP/1.1" 200 4583 "http://10.1.9.104:10725/home" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0"
[2024-10-23 16:40:20 +0800] [3310] [CRITICAL] WORKER TIMEOUT (pid:3316)
[2024-10-23 16:40:20 +0800] [3316] [ERROR] Error handling request /task_stats
Traceback (most recent call last):
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/gunicorn/workers/sync.py", line 134, in handle
self.handle_request(listener, req, client, addr)
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/gunicorn/workers/sync.py", line 177, in handle_request
respiter = self.wsgi(environ, resp.start_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/flask/app.py", line 2552, in __call__
return self.wsgi_app(environ, start_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/flask/app.py", line 2529, in wsgi_app
response = self.full_dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/airflow/www/auth.py", line 250, in decorated
return _has_access(
^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/airflow/www/auth.py", line 163, in _has_access
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/airflow/utils/session.py", line 97, in wrapper
return func(*args, session=session, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/airflow/www/views.py", line 1338, in task_stats
qry = session.execute(
^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 1717, in execute
result = conn._execute_20(statement, params or {}, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
self._handle_dbapi_exception(
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2138, in _handle_dbapi_exception
util.raise_(exc_info[1], with_traceback=exc_info[2])
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
result = self._query(query)
^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/pymysql/connections.py", line 563, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/pymysql/connections.py", line 825, in _read_query_result
result.read()
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/pymysql/connections.py", line 1199, in read
first_packet = self.connection._read_packet()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/pymysql/connections.py", line 744, in _read_packet
packet_header = self._read_bytes(4)
^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/pymysql/connections.py", line 782, in _read_bytes
data = self._rfile.read(num_bytes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/tsysmart/local/python/lib/python3.11/socket.py", line 706, in readinto
return self._sock.recv_into(b)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ttrs/airflow_dir/venv/lib/python3.11/site-packages/gunicorn/workers/base.py", line 204, in handle_abort
sys.exit(1)
SystemExit: 1
10.1.4.76 - - [23/Oct/2024:16:40:20 +0800] "POST /task_stats HTTP/1.1" 500 0 "-" "-"
[2024-10-23 16:40:20 +0800] [3316] [INFO] Worker exiting (pid: 3316)
[2024-10-23 16:40:21 +0800] [5542] [INFO] Booting worker with pid: 5542
10.1.4.76 - - [23/Oct/2024:16:40:22 +0800] "POST /next_run_datasets_summary HTTP/1.1" 200 2 "http://10.1.9.104:10725/home" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0"
10.1.4.76 - - [23/Oct/2024:16:40:22 +0800] "POST /last_dagruns HTTP/1.1" 200 8626 "http://10.1.9.104:10725/home" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0"
10.1.4.76 - - [23/Oct/2024:16:40:32 +0800] "POST /dag_stats HTTP/1.1" 200 4583 "http://10.1.9.104:10725/home" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0"
### What you think should happen instead?
_No response_
### How to reproduce
Is it possible that airlow, which runs locally, does not modify the underlying code and does not add components because there are too many DAGs (40 or so?)? Causing this error?
### Operating System
ubuntu
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
GitHub link: https://github.com/apache/airflow/discussions/43363
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] An error occurred in webservice， request： /task_stats [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
GitHub user shahar1 added a comment to the discussion: An error occurred in webservice， request： /task_stats
It is not as clear what the problem is and how to reproduce it with a minimal example.
Could you please elaborate?
GitHub link: https://github.com/apache/airflow/discussions/43363#discussioncomment-11044660
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] An error occurred in webservice， request： /task_stats [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: An error occurred in webservice， request： /task_stats
Converted it to discussion - since there is no reproduction and no explanation what happens. Maybe the author will have more information.
GitHub link: https://github.com/apache/airflow/discussions/43363#discussioncomment-11044670
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org