You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "Abhijith-Nagaraja (via GitHub)" <gi...@apache.org> on 2023/11/01 20:10:47 UTC
[I] Global Aysnc queries doesn't work with Athena [superset]
Abhijith-Nagaraja opened a new issue, #25827:
URL: https://github.com/apache/superset/issues/25827
A clear and concise description of what the bug is.
Tried to enable Global async queries. Was successfully able to start it with mysql RDS but aync_event API didn't even get fired from dashboard with Athena datasource
Here are my configurations
```
.
.
.
FEATURE_FLAGS: Dict[str, bool] = {
"ENABLE_TEMPLATE_PROCESSING": True,
"ENABLE_TEMPLATE_REMOVE_FILTERS": True,
"DASHBOARD_CACHE": True,
"ALERT_REPORTS": True,
"ALLOW_ADHOC_SUBQUERY": True,
"GLOBAL_ASYNC_QUERIES": True
}
#Caching configurations
FILTER_STATE_CACHE_CONFIG = {
"CACHE_TYPE": "RedisCache",
"CACHE_DEFAULT_TIMEOUT": 86400,
"CACHE_KEY_PREFIX": "superset_filter_cache",
"CACHE_REDIS_URL": "redis://superset_cache:6379/0"
}
EXPLORE_FORM_DATA_CACHE_CONFIG = {
"CACHE_TYPE": "RedisCache",
"CACHE_DEFAULT_TIMEOUT": 86400,
"CACHE_KEY_PREFIX": "superset_form_data_cache",
"CACHE_REDIS_URL": "redis://superset_cache:6379/2"
}
DATA_CACHE_CONFIG = {
"CACHE_TYPE": "SupersetMetastoreCache",
"CACHE_KEY_PREFIX": "superset_data_cache",
"CACHE_DEFAULT_TIMEOUT": 86400
}
CACHE_CONFIG = {
"CACHE_TYPE": "RedisCache",
"CACHE_DEFAULT_TIMEOUT": 86400,
"CACHE_KEY_PREFIX": "superset_filter_cache",
"CACHE_REDIS_URL": "redis://superset_cache:6379/1"
}
#GLOBAL ASYNC QUERIES CONFIG
GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
"port": 6379,
"host": "superset_cache",
"db": 3,
"ssl": False
}
GLOBAL_ASYNC_QUERIES_JWT_SECRET = "<FIX IT>"
GLOBAL_ASYNC_QUERIES_POLLING_DELAY = int(
timedelta(milliseconds=2000).total_seconds() * 1000
)
class CeleryConfig(object):
broker_url = "redis://superset_cache:6379/4"
imports = (
"superset.sql_lab",
"superset.tasks.scheduler",
)
result_backend = "redis://superset_cache:6379/4"
worker_prefetch_multiplier = 10
task_acks_late = True
task_annotations = {
"sql_lab.get_sql_results": {
"rate_limit": "100/s",
},
}
CELERY_CONFIG = CeleryConfig
```
`docker ps`
<img width="618" alt="Screenshot 2023-11-01 at 12 58 49 PM" src="https://github.com/apache/superset/assets/3592887/18f9d5b9-1535-4589-a975-c316b73b89d0">
`Async configuration for datasource in UI`
#### How to reproduce the bug
Step 1: Create two dashboards. One dashboard containing SQL datasource such as mysql and second one with Athena Data source.
Step 2: Enable Global Async Queries and corresponding setup
Step 3: Configure Celery and Redis
Step 4: Enable Asynchronous query execution from Database and Connections -> select datasource -> Edit -> Advanced -> check Asynchronous query execution
### Expected results
Charts within the dashboard should be loaded Asynchronously for both Mysql Rds datasource and Athena datasource
### Actual results
Charts within the dashboard loaded Asynchronously for Mysql Rds datasource but not for Athena datasource
#### Screenshots
If applicable, add screenshots to help explain your problem.
### Environment
(please complete the following information):
- browser type and version: `Chrome - 118.0.5993.117`
- superset version: `3.0`
- python library versions:
```
pyathena == 2.5.4
pyathenajdbc == 3.0.1
```
- any feature flags active:
`"GLOBAL_ASYNC_QUERIES": True`
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [ ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
Additional screenshots
Working with Mysql RDS datasource. As you can see in the below screenshot, async_event api got fired
Not working with Athena datasource. As you can see in the below screenshot, no async_even api is fired
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Global Aysnc queries doesn't work with Athena [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #25827:
URL: https://github.com/apache/superset/issues/25827#issuecomment-2040465318
Hi there! Async queries are still a bit "beta" even though there have been some improvements since this issue was filed. I'm curious if you're still facing this in Superset 3.1 or newer. Also curious if anyone else running GAQ (maybe @villebro ) have the ability to reproduce the issue as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org