You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/11/03 19:26:44 UTC
[GitHub] [superset] diogenes1oliveira opened a new issue, #22030: Queries from charts for async databases are being executed in the server instances, not the workers
diogenes1oliveira opened a new issue, #22030:
URL: https://github.com/apache/superset/issues/22030
Charts for databases with **Asynchronous query execution** aren't being executed in the worker nodes as expected.
#### How to reproduce the bug
1. Add a datasource in a setup with Async Queries configured via Celery;
2. Enable the option `Advanced > Performance > Asynchronous query execution`;
3. Add a sample chart in a dashboard.
4. Check the logs of both the webserver and the worker instances;
### Expected results
I expected the queries to be executed in the worker instance. For instance, when running through SQL Lab that's what happens. The unexpected behavior occurs when using dashboards.
### Actual results
We see the connection being established in the webserver instance, not the worker:
pod-app-hostname 2022-11-03 19:19:30,871 DEBUG sqlalchemy.pool.impl.NullPool Created new connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50>
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Created new connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50>
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SHOW VARIABLES LIKE 'sql_mode'
pod-app-hostname 2022-11-03 19:19:30,873 INFO sqlalchemy.engine.base.Engine SHOW VARIABLES LIKE 'sql_mode'
pod-app-hostname 2022-11-03 19:19:30,873 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,877 INFO sqlalchemy.engine.base.Engine SHOW VARIABLES LIKE 'lower_case_table_names'
pod-app-hostname 2022-11-03 19:19:30,878 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SHOW VARIABLES LIKE 'lower_case_table_names'
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SELECT DATABASE()
pod-app-hostname 2022-11-03 19:19:30,881 INFO sqlalchemy.engine.base.Engine SELECT DATABASE()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,882 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname 2022-11-03 19:19:30,884 INFO sqlalchemy.engine.base.Engine show collation where `Charset` = 'utf8mb4' and `Collation` = 'utf8mb4_bin'
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:show collation where `Charset` = 'utf8mb4' and `Collation` = 'utf8mb4_bin'
pod-app-hostname 2022-11-03 19:19:30,885 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,887 INFO sqlalchemy.engine.base.Engine SELECT CAST('test plain returns' AS CHAR(60)) AS anon_1
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SELECT CAST('test plain returns' AS CHAR(60)) AS anon_1
pod-app-hostname 2022-11-03 19:19:30,887 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,889 INFO sqlalchemy.engine.base.Engine SELECT CAST('test unicode returns' AS CHAR(60)) AS anon_1
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SELECT CAST('test unicode returns' AS CHAR(60)) AS anon_1
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,889 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:SELECT CAST('test collated returns' AS CHAR CHARACTER SET utf8mb4) COLLATE utf8mb4_bin AS anon_1
pod-app-hostname INFO:sqlalchemy.engine.base.Engine:()
pod-app-hostname 2022-11-03 19:19:30,891 INFO sqlalchemy.engine.base.Engine SELECT CAST('test collated returns' AS CHAR CHARACTER SET utf8mb4) COLLATE utf8mb4_bin AS anon_1
pod-app-hostname 2022-11-03 19:19:30,891 INFO sqlalchemy.engine.base.Engine ()
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50> checked out from pool
pod-app-hostname 2022-11-03 19:19:30,895 DEBUG sqlalchemy.pool.impl.NullPool Connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50> checked out from pool
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Pool pre-ping on connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50>
pod-app-hostname 2022-11-03 19:19:30,895 DEBUG sqlalchemy.pool.impl.NullPool Pool pre-ping on connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50>
pod-app-hostname 2022-11-03 19:19:30,901 DEBUG sqlalchemy.pool.impl.NullPool Connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50> being returned to pool
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50> being returned to pool
pod-app-hostname 2022-11-03 19:19:30,902 DEBUG sqlalchemy.pool.impl.NullPool Connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50> rollback-on-return
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50> rollback-on-return
pod-app-hostname 2022-11-03 19:19:30,903 DEBUG sqlalchemy.pool.impl.NullPool Closing connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50>
pod-app-hostname DEBUG:sqlalchemy.pool.impl.NullPool:Closing connection <_mysql.connection open to '1.2.3.4' at 0x55beb238bc50>
#### Screenshots
If applicable, add screenshots to help explain your problem.
### Environment
(please complete the following information):
- browser type and version: Google Chrome 107.0.5304.87
- superset version: `2.0.0`
- python version: `Python 3.8.12`
- node.js version: `Not installed in the Docker instance`
- any feature flags active: ```
FEATURE_FLAGS = {
'ENABLE_TEMPLATE_PROCESSING': True,
}
```
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [ x ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [ x ] I have reproduced the issue with at least the latest released version of superset.
- [ x ] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
I'm using an image derived from `apache/superset:2.0.0` and tried to connect both to Hive and to MySQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] unnyns-307 commented on issue #22030: Queries from charts for async databases are being executed in the server instances, not the workers
Posted by "unnyns-307 (via GitHub)" <gi...@apache.org>.
unnyns-307 commented on issue #22030:
URL: https://github.com/apache/superset/issues/22030#issuecomment-1567711283
Hi, we are facing the same issue. @diogenes1oliveira did you find any fix or workaround?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Queries from charts for async databases are being executed in the server instances, not the workers [superset]
Posted by "dingbat (via GitHub)" <gi...@apache.org>.
dingbat commented on issue #22030:
URL: https://github.com/apache/superset/issues/22030#issuecomment-1747420175
Enabling AQE for a database only applies to SQL Lab - for charts you'll need to enable the `GLOBAL_ASYNC_QUERIES` feature flag and add corresponding config (see docs here: https://github.com/apache/superset/blob/master/CONTRIBUTING.md#async-chart-queries)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Queries from charts for async databases are being executed in the server instances, not the workers [superset]
Posted by "shauryagoel (via GitHub)" <gi...@apache.org>.
shauryagoel commented on issue #22030:
URL: https://github.com/apache/superset/issues/22030#issuecomment-1945476398
Hi @dingbat, is there any doc or guide which shows how to use `GLOBAL_ASYNC_QUERIES`? I tried setting this to true, but, I started getting getting jwt token error. After setting `GLOBAL_ASYNC_QUERIES_JWT_SECRET` to some random 32 bit number, I am getting these types of logs continuously-
`10.160.15.225 - - [15/Feb/2024:06:59:18 +0000] "GET /api/v1/async_event/ HTTP/1.1" 403 24 "http://IP/dashboard/45/?native_filters_key=RkL3s6EjCpRaxau_afgK1VBnmRh5Qx1pfj8z4UlRXoDyVCKAq6IWEhlVxvXEbzu5" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org