You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/04/22 04:00:49 UTC
[GitHub] [superset] saLeox opened a new issue, #19816: Add distributed lock for concurrent query before retrieve cache
saLeox opened a new issue, #19816:
URL: https://github.com/apache/superset/issues/19816
It's known that the Redis cache is already applied in query for query_context and viz, but there is a situation that makes it doesn’t work well, that’s when multi concurrent query requests to one dashboard or chart, come in at the same time, from different users. It will cause duplicated query request to our query engine, because the cache is not ready yet. As a result, it brings burden to cluster and waste resources.
Since the superset is natively distributed, and the async task is supposed to run in multiple celery works, we come up with an assumption that, maybe the distributed lock can help, which should be acquired before cache retrieve and released after query complete, to make the queries with the same cache key execute in a serial way, instead of a concurrent way.
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rusackas closed issue #19816: Add distributed lock for concurrent query before retrieve cache
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #19816: Add distributed lock for concurrent query before retrieve cache
URL: https://github.com/apache/superset/issues/19816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rusackas commented on issue #19816: Add distributed lock for concurrent query before retrieve cache
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #19816:
URL: https://github.com/apache/superset/issues/19816#issuecomment-1572320059
The `GLOBAL_ASYNC_QUERIES` feature is designed to address this very issue. I would recommend experimenting with and stabilizing/supporting that feature. If you have an alternate design/approach that might be simpler, I think we'd be open to a [Superset Improvement Proposal (SIP)](https://github.com/apache/superset/issues/5602) or taking this to a new [Ideas thread](https://github.com/apache/superset/discussions/categories/ideas) in our [Github Discussions](https://github.com/apache/superset/discussions) area.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org