You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "gvnn-gov (via GitHub)" <gi...@apache.org> on 2025/07/24 07:01:40 UTC
[I] column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function [airflow]
gvnn-gov opened a new issue, #53695:
URL: https://github.com/apache/airflow/issues/53695
### Apache Airflow version
3.0.3
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Visiting the `/events` page I'm presented with `500 Internal Server Error`.
Below the error from the server logs
```
INFO: 10.52.2.155:33530 - "GET /api/v2/eventLogs?limit=50&offset=0&order_by=-when HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.GroupingError: column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function
LINE 2: FROM (SELECT log.id AS id, log.dttm AS dttm, log.dag_id AS d...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 29, in __call__
await responder(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 126, in __call__
await super().__call__(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 46, in __call__
await self.app(scope, receive, self.send_with_compression)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 176, in __call__
with recv_stream, send_stream, collapse_excgroups():
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
self.gen.throw(value)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_utils.py", line 82, in collapse_excgroups
raise exc
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 178, in __call__
response = await self.dispatch_func(request, call_next)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/core_api/middleware.py", line 28, in dispatch
response = await call_next(request)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 156, in call_next
raise app_exc
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 141, in coro
await self.app(scope, receive_or_disconnect, send_no_error)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 714, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 734, in app
await route.handle(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/fastapi/routing.py", line 214, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/starlette/concurrency.py", line 37, in run_in_threadpool
return await anyio.to_thread.run_sync(func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/core_api/routes/public/event_logs.py", line 120, in get_event_logs
event_logs_select, total_entries = paginated_select(
^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py", line 99, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/common/db/common.py", line 172, in paginated_select
total_entries = get_query_count(statement, session=session)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/db.py", line 1405, in get_query_count
return session.scalar(count_stmt)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1747, in scalar
return self.execute(
^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1717, in execute
result = conn._execute_20(statement, params or {}, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
self._handle_dbapi_exception(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapi_exception
util.raise_(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
self.dialect.do_execute(
File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.GroupingError) column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function
LINE 2: FROM (SELECT log.id AS id, log.dttm AS dttm, log.dag_id AS d...
^
[SQL: SELECT count(*) AS count_1
FROM (SELECT log.id AS id, log.dttm AS dttm, log.dag_id AS dag_id, log.task_id AS task_id, log.map_index AS map_index, log.event AS event, log.logical_date AS logical_date, log.run_id AS run_id, log.owner AS owner, log.owner_display_name AS owner_display_name, log.extra AS extra, log.try_number AS try_number
FROM log GROUP BY log.id) AS anon_1]
(Background on this error at: https://sqlalche.me/e/14/f405)
```
I suspect that the issue is around this line https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/api_fastapi/core_api/routes/public/event_logs.py#L119
### What you think should happen instead?
_No response_
### How to reproduce
We have an instance deployed using the latest helm char and Airflow version 3.0.2. We get error 500 every time we call `/api/v2/eventLogs`.
### Operating System
Debian GNU/Linux 12 (bookworm)
### Versions of Apache Airflow Providers
```apache-airflow-providers-amazon==9.8.0
apache-airflow-providers-celery==3.11.0
apache-airflow-providers-cncf-kubernetes==10.5.0
apache-airflow-providers-common-compat==1.7.0
apache-airflow-providers-common-io==1.6.0
apache-airflow-providers-common-messaging==1.0.3
apache-airflow-providers-common-sql==1.27.1
apache-airflow-providers-docker==4.4.0
apache-airflow-providers-elasticsearch==6.3.0
apache-airflow-providers-fab==2.2.1
apache-airflow-providers-ftp==3.13.0
apache-airflow-providers-git==0.0.2
apache-airflow-providers-google==15.1.0
apache-airflow-providers-grpc==3.8.0
apache-airflow-providers-hashicorp==4.2.0
apache-airflow-providers-http==5.3.0
apache-airflow-providers-microsoft-azure==12.4.0
apache-airflow-providers-mysql==6.3.0
apache-airflow-providers-odbc==4.10.0
apache-airflow-providers-openlineage==2.3.0
apache-airflow-providers-postgres==6.2.0
apache-airflow-providers-redis==4.1.0
apache-airflow-providers-sendgrid==4.1.0
apache-airflow-providers-sftp==5.3.0
apache-airflow-providers-slack==9.1.0
apache-airflow-providers-smtp==2.1.0
apache-airflow-providers-snowflake==6.3.1
apache-airflow-providers-ssh==4.1.0
apache-airflow-providers-standard==1.2.0```
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function [airflow]
Posted by "gvnn-gov (via GitHub)" <gi...@apache.org>.
gvnn-gov commented on issue #53695:
URL: https://github.com/apache/airflow/issues/53695#issuecomment-3124835600
And this is what the query looks like when executed in pgAdmin
<img width="974" height="470" alt="Image" src="https://github.com/user-attachments/assets/87c15336-b6a8-4f5c-9662-b3a7bc9eda49" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function [airflow]
Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on issue #53695:
URL: https://github.com/apache/airflow/issues/53695#issuecomment-3125795359
Cc @vatsrahul1001 can you please try to reproduce this issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #53695:
URL: https://github.com/apache/airflow/issues/53695#issuecomment-3112289368
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function [airflow]
Posted by "gvnn-gov (via GitHub)" <gi...@apache.org>.
gvnn-gov commented on issue #53695:
URL: https://github.com/apache/airflow/issues/53695#issuecomment-3124809752
> What database are you using? I tried both postgres and mysql and I'm not able to reproduce this.
It's a PostgreSQL 15.12, hosted in Azure (Azure Database for PostgreSQL flexible server). What I didn't mention, and maybe it's the real problem, this was an upgrade from `2.10.5` to `3.0.2`. The migrations ran perfectly, but maybe there were some leftovers?
This is the current table definition
```sql
CREATE TABLE IF NOT EXISTS public.log
(
id integer NOT NULL DEFAULT nextval('log_id_seq'::regclass),
dttm timestamp with time zone,
dag_id character varying(250) COLLATE pg_catalog."default",
task_id character varying(250) COLLATE pg_catalog."default",
event character varying(60) COLLATE pg_catalog."default",
logical_date timestamp with time zone,
owner character varying(500) COLLATE pg_catalog."default",
extra text COLLATE pg_catalog."default",
map_index integer,
owner_display_name character varying(500) COLLATE pg_catalog."default",
run_id character varying(250) COLLATE pg_catalog."default",
try_number integer
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53695:
URL: https://github.com/apache/airflow/issues/53695#issuecomment-3118238744
What database are you using?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun closed issue #53695: column "log.dttm" must appear in the GROUP BY clause or be used in an aggregate function
URL: https://github.com/apache/airflow/issues/53695
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org