You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2023/01/11 17:50:32 UTC

[GitHub] [superset] reidab opened a new pull request, #22687: docs(caching): Restructure and improve caching docs

reidab opened a new pull request, #22687:
URL: https://github.com/apache/superset/pull/22687

   <!---
   Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/
   Example:
   fix(dashboard): load charts correctly
   -->
   
   ### SUMMARY
   
   Improves caching documentation. This PR attempts to reorder and rewrite existing caching documentation for improved clarity. 
   
   Specifically:
   * Being more explicit about the required dictionary format for cache configuration and adding a simple redis example.
   * Splitting `RESULTS_BACKEND` into its own section and clarifying that it is _not_ configured using a flask-caching dictionary.
   * Adding a header to call out the additional python dependencies needed for for redis and memcache.
   * Breaking out information on the fallback `SupersetMetastoreCache` into its own section.
   
   ### TESTING INSTRUCTIONS
   
   Read the new docs! Do the accurately describe cache configuration in a clear manner?
   
   ### ADDITIONAL INFORMATION
   <!--- Check any relevant boxes with "x" -->
   <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
   - [ ] Has associated issue:
   - [ ] Required feature flags:
   - [ ] Changes UI
   - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
     - [ ] Migration is atomic, supports rollback & is backwards-compatible
     - [ ] Confirm DB migration upgrade and downgrade tested
     - [ ] Runtime estimates and downtime expectations provided
   - [ ] Introduces new feature or API
   - [ ] Removes existing feature or API
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

[GitHub] [superset] villebro commented on a diff in pull request #22687: docs(caching): Restructure and improve caching docs

Posted by GitBox <gi...@apache.org>.
villebro commented on code in PR #22687:
URL: https://github.com/apache/superset/pull/22687#discussion_r1067923908


##########
docs/docs/installation/cache.mdx:
##########
@@ -7,19 +7,46 @@ version: 1
 
 ## Caching
 
-Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Configuring caching is as easy as providing a custom cache config in your
-`superset_config.py` that complies with [the Flask-Caching specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
-Flask-Caching supports various caching backends, including Redis, Memcached, SimpleCache (in-memory), or the
-local filesystem. Custom cache backends are also supported. See [here](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) for specifics.
-The following cache configurations can be customized:
-- Metadata cache (optional): `CACHE_CONFIG`
-- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG`
-- SQL Lab query results (optional): `RESULTS_BACKEND`. See [Async Queries via Celery](/docs/installation/async-queries-celery) for details
+Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Flask-Caching supports various caching backends, including Redis (recommended), Memcached, SimpleCache (in-memory), or the local filesystem. [Custom cache backends](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) are also supported.
+
+Caching can be configured by providing a dictionaries in
+`superset_config.py` that comply with [the Flask-Caching config specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
+
+The following cache configurations can be customized in this way:
 - Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`.
 - Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG`
+- Metadata cache (optional): `CACHE_CONFIG`
+- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG`
+
+For example, to configure the filter state cache using redis:
+
+```python
+FILTER_STATE_CACHE_CONFIG = {
+    'CACHE_TYPE': 'RedisCache',
+    'CACHE_DEFAULT_TIMEOUT': 86400,
+    'CACHE_KEY_PREFIX': 'superset_filter_cache',
+    'CACHE_REDIS_URL': 'redis://localhost:6379/0'
+}
+```
+
+For chart data, Superset goes up a “timeout search path”, from the chart's configuration
+to the dataset’s, the database’s, then ultimately falls back to the global default
+defined in `DATA_CACHE_CONFIG`.

Review Comment:
   @reidab what this is trying to convey is the fact that you can override the cache timeout in the chart, dataset and database (checked in that order), and if none of those is defined, it will use the default cache timeout defined in the cache config.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

[GitHub] [superset] reidab commented on a diff in pull request #22687: docs(caching): Restructure and improve caching docs

Posted by GitBox <gi...@apache.org>.
reidab commented on code in PR #22687:
URL: https://github.com/apache/superset/pull/22687#discussion_r1068795722


##########
docs/docs/installation/cache.mdx:
##########
@@ -7,19 +7,46 @@ version: 1
 
 ## Caching
 
-Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Configuring caching is as easy as providing a custom cache config in your
-`superset_config.py` that complies with [the Flask-Caching specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
-Flask-Caching supports various caching backends, including Redis, Memcached, SimpleCache (in-memory), or the
-local filesystem. Custom cache backends are also supported. See [here](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) for specifics.
-The following cache configurations can be customized:
-- Metadata cache (optional): `CACHE_CONFIG`
-- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG`
-- SQL Lab query results (optional): `RESULTS_BACKEND`. See [Async Queries via Celery](/docs/installation/async-queries-celery) for details
+Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Flask-Caching supports various caching backends, including Redis (recommended), Memcached, SimpleCache (in-memory), or the local filesystem. [Custom cache backends](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) are also supported.
+
+Caching can be configured by providing a dictionaries in
+`superset_config.py` that comply with [the Flask-Caching config specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
+
+The following cache configurations can be customized in this way:
 - Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`.
 - Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG`
+- Metadata cache (optional): `CACHE_CONFIG`
+- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG`
+
+For example, to configure the filter state cache using redis:
+
+```python
+FILTER_STATE_CACHE_CONFIG = {
+    'CACHE_TYPE': 'RedisCache',
+    'CACHE_DEFAULT_TIMEOUT': 86400,
+    'CACHE_KEY_PREFIX': 'superset_filter_cache',
+    'CACHE_REDIS_URL': 'redis://localhost:6379/0'
+}
+```
+
+For chart data, Superset goes up a “timeout search path”, from the chart's configuration
+to the dataset’s, the database’s, then ultimately falls back to the global default
+defined in `DATA_CACHE_CONFIG`.

Review Comment:
   @villebro Thanks! That helps to clarify it. I hadn't understood that this was referring to just the timeout, but it makes sense now. I rewrote it a bit to be more explicit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

[GitHub] [superset] villebro merged pull request #22687: docs(caching): Restructure and improve caching docs

Posted by GitBox <gi...@apache.org>.
villebro merged PR #22687:
URL: https://github.com/apache/superset/pull/22687


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

[GitHub] [superset] reidab commented on a diff in pull request #22687: docs(caching): Restructure and improve caching docs

Posted by GitBox <gi...@apache.org>.
reidab commented on code in PR #22687:
URL: https://github.com/apache/superset/pull/22687#discussion_r1067292031


##########
docs/docs/installation/cache.mdx:
##########
@@ -7,19 +7,46 @@ version: 1
 
 ## Caching
 
-Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Configuring caching is as easy as providing a custom cache config in your
-`superset_config.py` that complies with [the Flask-Caching specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
-Flask-Caching supports various caching backends, including Redis, Memcached, SimpleCache (in-memory), or the
-local filesystem. Custom cache backends are also supported. See [here](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) for specifics.
-The following cache configurations can be customized:
-- Metadata cache (optional): `CACHE_CONFIG`
-- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG`
-- SQL Lab query results (optional): `RESULTS_BACKEND`. See [Async Queries via Celery](/docs/installation/async-queries-celery) for details
+Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Flask-Caching supports various caching backends, including Redis (recommended), Memcached, SimpleCache (in-memory), or the local filesystem. [Custom cache backends](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) are also supported.
+
+Caching can be configured by providing a dictionaries in
+`superset_config.py` that comply with [the Flask-Caching config specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
+
+The following cache configurations can be customized in this way:
 - Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`.
 - Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG`
+- Metadata cache (optional): `CACHE_CONFIG`
+- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG`
+
+For example, to configure the filter state cache using redis:
+
+```python
+FILTER_STATE_CACHE_CONFIG = {
+    'CACHE_TYPE': 'RedisCache',
+    'CACHE_DEFAULT_TIMEOUT': 86400,
+    'CACHE_KEY_PREFIX': 'superset_filter_cache',
+    'CACHE_REDIS_URL': 'redis://localhost:6379/0'
+}
+```
+
+For chart data, Superset goes up a “timeout search path”, from the chart's configuration
+to the dataset’s, the database’s, then ultimately falls back to the global default
+defined in `DATA_CACHE_CONFIG`.

Review Comment:
   I really have no idea what this means and I wouldn't guess most first-time users would either. I maintained it because it was in the existing docs, but I think it's either in need of further clarification by someone who understands it or should be removed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

[GitHub] [superset] villebro commented on a diff in pull request #22687: docs(caching): Restructure and improve caching docs

Posted by GitBox <gi...@apache.org>.
villebro commented on code in PR #22687:
URL: https://github.com/apache/superset/pull/22687#discussion_r1068957525


##########
docs/docs/installation/cache.mdx:
##########
@@ -7,19 +7,46 @@ version: 1
 
 ## Caching
 
-Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Configuring caching is as easy as providing a custom cache config in your
-`superset_config.py` that complies with [the Flask-Caching specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
-Flask-Caching supports various caching backends, including Redis, Memcached, SimpleCache (in-memory), or the
-local filesystem. Custom cache backends are also supported. See [here](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) for specifics.
-The following cache configurations can be customized:
-- Metadata cache (optional): `CACHE_CONFIG`
-- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG`
-- SQL Lab query results (optional): `RESULTS_BACKEND`. See [Async Queries via Celery](/docs/installation/async-queries-celery) for details
+Superset uses [Flask-Caching](https://flask-caching.readthedocs.io/) for caching purposes. Flask-Caching supports various caching backends, including Redis (recommended), Memcached, SimpleCache (in-memory), or the local filesystem. [Custom cache backends](https://flask-caching.readthedocs.io/en/latest/#custom-cache-backends) are also supported.
+
+Caching can be configured by providing a dictionaries in
+`superset_config.py` that comply with [the Flask-Caching config specifications](https://flask-caching.readthedocs.io/en/latest/#configuring-flask-caching).
+
+The following cache configurations can be customized in this way:
 - Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`.
 - Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG`
+- Metadata cache (optional): `CACHE_CONFIG`
+- Charting data queried from datasets (optional): `DATA_CACHE_CONFIG`
+
+For example, to configure the filter state cache using redis:
+
+```python
+FILTER_STATE_CACHE_CONFIG = {
+    'CACHE_TYPE': 'RedisCache',
+    'CACHE_DEFAULT_TIMEOUT': 86400,
+    'CACHE_KEY_PREFIX': 'superset_filter_cache',
+    'CACHE_REDIS_URL': 'redis://localhost:6379/0'
+}
+```
+
+For chart data, Superset goes up a “timeout search path”, from the chart's configuration
+to the dataset’s, the database’s, then ultimately falls back to the global default
+defined in `DATA_CACHE_CONFIG`.

Review Comment:
   @reidab awesome, thanks for helping clarify this!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org