You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/03/17 08:12:21 UTC
[GitHub] [superset] villebro opened a new pull request #19232: [WIP] feat(key-value): add superset cache
villebro opened a new pull request #19232:
URL: https://github.com/apache/superset/pull/19232
### SUMMARY
Add custom `Flask-Caching` backend that leverages the new `key_value` table in the Superset metadata database. This will make it possible to use the metadata database as a cache.
TODO:
- cleanup
- add tests
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ef1984e) into [master](https://codecov.io/gh/apache/superset/commit/51061f0d672abca29f84943acb16a37403f25c2e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (51061f0) will **increase** coverage by `0.03%`.
> The diff coverage is `89.58%`.
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.76% 66.79% +0.03%
==========================================
Files 1670 1672 +2
Lines 64392 64488 +96
Branches 6499 6501 +2
==========================================
+ Hits 42991 43076 +85
- Misses 19718 19728 +10
- Partials 1683 1684 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.47% <5.88%> (-0.06%)` | :arrow_down: |
| mysql | `82.00% <89.58%> (+0.03%)` | :arrow_up: |
| postgres | `?` | |
| presto | `52.32% <5.88%> (-0.06%)` | :arrow_down: |
| python | `82.42% <89.58%> (-0.02%)` | :arrow_down: |
| sqlite | `81.80% <89.58%> (+0.03%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <0.00%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `91.83% <75.00%> (+2.36%)` | :arrow_up: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NhY2hlLnB5) | `100.00% <100.00%> (ø)` | |
| [superset/utils/cache\_manager.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvY2FjaGVfbWFuYWdlci5weQ==) | `100.00% <100.00%> (+6.25%)` | :arrow_up: |
| ... and [35 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [51061f0...ef1984e](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
michael-s-molina commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830019202
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
Review comment:
Can we use `_prune` instead of `_purge`? If you check Flask-Caching backends, that's the name they are using.
##########
File path: superset-frontend/cypress-base/cypress/integration/dashboard/key_value.test.ts
##########
@@ -27,7 +27,7 @@ interface QueryString {
native_filters_key: string;
}
-xdescribe('nativefiler url param key', () => {
+describe('nativefiler url param key', () => {
Review comment:
I'm not sure this problem is related to the cache. Although it's the `key_value.test` that is failing, you can see that the error is happening in `dashboardHelper/getChartGridComponent`
##########
File path: superset/utils/cache_manager.py
##########
@@ -40,27 +38,27 @@ def _init_cache(
) -> None:
cache_config = app.config[cache_config_key]
cache_type = cache_config.get("CACHE_TYPE")
- if app.debug and cache_type is None:
- cache_threshold = cache_config.get("CACHE_THRESHOLD", math.inf)
+ if required and cache_type in (None, "SupersetCache"):
+ if cache_type is None:
+ logger.warning(
+ "Falling back to built-in key-value cache for the following "
Review comment:
Maybe "Falling back to the built-in cache, that stores data in the metadata database, for the following..."
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ try:
Review comment:
Can't we just use the update command?
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ try:
+ DeleteKeyValueCommand(
+ resource=RESOURCE, key_type=KEY_TYPE, key=self.get_key(key),
+ ).run()
+ except KeyValueCreateFailedError:
+ pass
+ return self.add(key, value, timeout)
+
+ def add(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.create import CreateKeyValueCommand
+
+ try:
+ CreateKeyValueCommand(
+ resource=RESOURCE,
+ value=value,
+ key_type=KEY_TYPE,
+ key=self.get_key(key),
+ expires_on=self.get_expiry(timeout),
+ ).run()
+ self._purge()
+ return True
+ except KeyValueCreateFailedError:
+ return False
+
+ def get(self, key: str) -> Any:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.get import GetKeyValueCommand
+
+ return GetKeyValueCommand(
+ resource=RESOURCE, key_type=KEY_TYPE, key=self.get_key(key),
+ ).run()
+
+ def has(self, key: str) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.get import GetKeyValueCommand
+
+ entry = GetKeyValueCommand(
Review comment:
Maybe reference `get` here so you can keep the logic in just one place?
##########
File path: docs/docs/installation/cache.mdx
##########
@@ -18,10 +18,17 @@ The following cache configurations can be customized:
- Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`.
- Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG`
-Please note, that Dashboard and Explore caching is required. When running Superset in debug mode, both Explore and Dashboard caches will default to `SimpleCache`;
-However, trying to run Superset in non-debug mode without defining a cache for these will cause the application to fail on startup. When running
-superset in single-worker mode, any cache backend is supported. However, when running Superset in on a multi-worker setup, a dedicated cache is required. For this
-we recommend using either Redis or Memcached:
+Please note, that Dashboard and Explore caching is required. If these caches are undefined, Superset falls back to using a built-in cache that stores data
+in the metadata database. While it is recommended to use a dedicated cache, the built-in cache can also be used to cache other data.
+For example, to use the built-in cache to store chart data, use the following config:
+
+```python
+DATA_CACHE_CONFIG = {
+ "CACHE_TYPE": "SupersetCache",
+ "CACHE_KEY_PREFIX": "superset_results", # make sure this string is unique to avoid collisions
+ "CACHE_DEFAULT_TIMEOUT": 86400, # 60 seconds * 60 minutes * 24 hours
Review comment:
Maybe use `int(timedelta(hours=24).total_seconds())`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (58fd5ba) into [master](https://codecov.io/gh/apache/superset/commit/d645579cdd64b7fe7f9dde4a8da000dd7db51ce9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d645579) will **decrease** coverage by `0.09%`.
> The diff coverage is `90.13%`.
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
- Coverage 66.53% 66.44% -0.10%
==========================================
Files 1667 1670 +3
Lines 64360 64500 +140
Branches 6493 6493
==========================================
+ Hits 42824 42858 +34
- Misses 19854 19960 +106
Partials 1682 1682
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.54% <8.21%> (-0.11%)` | :arrow_down: |
| mysql | `?` | |
| postgres | `?` | |
| presto | `52.38% <8.21%> (-0.11%)` | :arrow_down: |
| python | `81.70% <90.13%> (-0.26%)` | :arrow_down: |
| sqlite | `81.35% <90.13%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <66.66%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `93.87% <83.33%> (+4.40%)` | :arrow_up: |
| [superset/key\_value/commands/upsert.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3Vwc2VydC5weQ==) | `89.79% <89.79%> (ø)` | |
| [superset/extensions/metastore\_cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXh0ZW5zaW9ucy9tZXRhc3RvcmVfY2FjaGUucHk=) | `98.18% <98.18%> (ø)` | |
| ... and [23 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d645579...58fd5ba](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4e9c2f6) into [master](https://codecov.io/gh/apache/superset/commit/51061f0d672abca29f84943acb16a37403f25c2e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (51061f0) will **increase** coverage by `0.03%`.
> The diff coverage is `87.09%`.
> :exclamation: Current head 4e9c2f6 differs from pull request most recent head 78dfe29. Consider uploading reports for the commit 78dfe29 to get more accurate results
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.76% 66.79% +0.03%
==========================================
Files 1670 1672 +2
Lines 64392 64488 +96
Branches 6499 6501 +2
==========================================
+ Hits 42991 43076 +85
- Misses 19718 19728 +10
- Partials 1683 1684 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.47% <20.43%> (-0.06%)` | :arrow_down: |
| mysql | `82.00% <87.60%> (+0.03%)` | :arrow_up: |
| postgres | `?` | |
| presto | `52.32% <20.43%> (-0.06%)` | :arrow_down: |
| python | `82.42% <87.60%> (-0.02%)` | :arrow_down: |
| sqlite | `81.80% <87.60%> (+0.03%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ns/legacy-plugin-chart-horizon/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LWhvcml6b24vc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...acy-plugin-chart-paired-t-test/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhaXJlZC10LXRlc3Qvc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gin-chart-parallel-coordinates/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhcmFsbGVsLWNvb3JkaW5hdGVzL3NyYy9jb250cm9sUGFuZWwudHM=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...legacy-plugin-chart-partition/src/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhcnRpdGlvbi9zcmMvY29udHJvbFBhbmVsLnRzeA==) | `33.33% <ø> (+8.33%)` | :arrow_up: |
| [...egacy-plugin-chart-pivot-table/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBpdm90LXRhYmxlL3NyYy9jb250cm9sUGFuZWwudHM=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gins/legacy-plugin-chart-rose/src/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXJvc2Uvc3JjL2NvbnRyb2xQYW5lbC50c3g=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...ns/legacy-plugin-chart-treemap/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXRyZWVtYXAvc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gacy-preset-chart-nvd3/src/DistBar/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcHJlc2V0LWNoYXJ0LW52ZDMvc3JjL0Rpc3RCYXIvY29udHJvbFBhbmVsLnRz) | `11.11% <ø> (+1.11%)` | :arrow_up: |
| [...chart-echarts/src/Timeseries/Area/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvVGltZXNlcmllcy9BcmVhL2NvbnRyb2xQYW5lbC50c3g=) | `40.00% <ø> (+6.66%)` | :arrow_up: |
| [...charts/src/Timeseries/Regular/Bar/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvVGltZXNlcmllcy9SZWd1bGFyL0Jhci9jb250cm9sUGFuZWwudHN4) | `40.00% <ø> (+6.66%)` | :arrow_up: |
| ... and [37 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [51061f0...78dfe29](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (58fd5ba) into [master](https://codecov.io/gh/apache/superset/commit/d645579cdd64b7fe7f9dde4a8da000dd7db51ce9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d645579) will **increase** coverage by `0.05%`.
> The diff coverage is `90.13%`.
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.53% 66.59% +0.05%
==========================================
Files 1667 1670 +3
Lines 64360 64500 +140
Branches 6493 6493
==========================================
+ Hits 42824 42955 +131
- Misses 19854 19863 +9
Partials 1682 1682
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.54% <8.21%> (-0.11%)` | :arrow_down: |
| mysql | `81.53% <90.13%> (+0.05%)` | :arrow_up: |
| postgres | `81.58% <90.13%> (+0.05%)` | :arrow_up: |
| presto | `52.38% <8.21%> (-0.11%)` | :arrow_down: |
| python | `82.00% <90.13%> (+0.05%)` | :arrow_up: |
| sqlite | `81.35% <90.13%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <66.66%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `93.87% <83.33%> (+4.40%)` | :arrow_up: |
| [superset/key\_value/commands/upsert.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3Vwc2VydC5weQ==) | `89.79% <89.79%> (ø)` | |
| [superset/extensions/metastore\_cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXh0ZW5zaW9ucy9tZXRhc3RvcmVfY2FjaGUucHk=) | `98.18% <98.18%> (ø)` | |
| ... and [4 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d645579...58fd5ba](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4e9c2f6) into [master](https://codecov.io/gh/apache/superset/commit/51061f0d672abca29f84943acb16a37403f25c2e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (51061f0) will **decrease** coverage by `0.09%`.
> The diff coverage is `87.09%`.
> :exclamation: Current head 4e9c2f6 differs from pull request most recent head 78dfe29. Consider uploading reports for the commit 78dfe29 to get more accurate results
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
- Coverage 66.76% 66.66% -0.10%
==========================================
Files 1670 1672 +2
Lines 64392 64488 +96
Branches 6499 6501 +2
==========================================
Hits 42991 42991
- Misses 19718 19813 +95
- Partials 1683 1684 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.47% <20.43%> (-0.06%)` | :arrow_down: |
| mysql | `?` | |
| postgres | `?` | |
| presto | `52.32% <20.43%> (-0.06%)` | :arrow_down: |
| python | `82.15% <87.60%> (-0.29%)` | :arrow_down: |
| sqlite | `81.80% <87.60%> (+0.03%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ns/legacy-plugin-chart-horizon/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LWhvcml6b24vc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...acy-plugin-chart-paired-t-test/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhaXJlZC10LXRlc3Qvc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gin-chart-parallel-coordinates/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhcmFsbGVsLWNvb3JkaW5hdGVzL3NyYy9jb250cm9sUGFuZWwudHM=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...legacy-plugin-chart-partition/src/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhcnRpdGlvbi9zcmMvY29udHJvbFBhbmVsLnRzeA==) | `33.33% <ø> (+8.33%)` | :arrow_up: |
| [...egacy-plugin-chart-pivot-table/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBpdm90LXRhYmxlL3NyYy9jb250cm9sUGFuZWwudHM=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gins/legacy-plugin-chart-rose/src/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXJvc2Uvc3JjL2NvbnRyb2xQYW5lbC50c3g=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...ns/legacy-plugin-chart-treemap/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXRyZWVtYXAvc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gacy-preset-chart-nvd3/src/DistBar/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcHJlc2V0LWNoYXJ0LW52ZDMvc3JjL0Rpc3RCYXIvY29udHJvbFBhbmVsLnRz) | `11.11% <ø> (+1.11%)` | :arrow_up: |
| [...chart-echarts/src/Timeseries/Area/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvVGltZXNlcmllcy9BcmVhL2NvbnRyb2xQYW5lbC50c3g=) | `40.00% <ø> (+6.66%)` | :arrow_up: |
| [...charts/src/Timeseries/Regular/Bar/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvVGltZXNlcmllcy9SZWd1bGFyL0Jhci9jb250cm9sUGFuZWwudHN4) | `40.00% <ø> (+6.66%)` | :arrow_up: |
| ... and [50 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [51061f0...78dfe29](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830204250
##########
File path: tests/integration_tests/key_value/cache_test.py
##########
@@ -0,0 +1,76 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+from datetime import datetime, timedelta
+from typing import TYPE_CHECKING
+from uuid import UUID
+
+import pytest
+from flask.ctx import AppContext
+from freezegun import freeze_time
+
+if TYPE_CHECKING:
+ from superset.key_value.cache import SupersetCache
+
+FIRST_KEY = "foo"
+FIRST_KEY_INITIAL_VALUE = {"foo": "bar"}
+FIRST_KEY_UPDATED_VALUE = "foo"
+
+SECOND_KEY = "baz"
+SECOND_VALUE = "qwerty"
+
+
+@pytest.fixture
+def cache() -> SupersetCache:
+ from superset.key_value.cache import SupersetCache
+
+ return SupersetCache(
+ namespace=UUID("ee173d1b-ccf3-40aa-941c-985c15224496"), default_timeout=600,
+ )
+
+
+def test_caching_flow(app_context: AppContext, cache: SupersetCache) -> None:
+ assert cache.has(FIRST_KEY) is False
+ assert cache.add(FIRST_KEY, FIRST_KEY_INITIAL_VALUE) is True
+ assert cache.has(FIRST_KEY) is True
+ cache.set(SECOND_KEY, SECOND_VALUE)
+ assert cache.get(FIRST_KEY) == FIRST_KEY_INITIAL_VALUE
+ assert cache.get(SECOND_KEY) == SECOND_VALUE
+ assert cache.add(FIRST_KEY, FIRST_KEY_UPDATED_VALUE) is False
+ assert cache.get(FIRST_KEY) == FIRST_KEY_INITIAL_VALUE
+ assert cache.set(FIRST_KEY, FIRST_KEY_UPDATED_VALUE) == True
+ assert cache.get(FIRST_KEY) == FIRST_KEY_UPDATED_VALUE
+ cache.delete(FIRST_KEY)
+ assert cache.has(FIRST_KEY) is False
+ assert cache.get(FIRST_KEY) is None
+ assert cache.has(SECOND_KEY)
+ assert cache.get(SECOND_KEY) == SECOND_VALUE
+
+
+def test_expiry(app_context: AppContext, cache: SupersetCache) -> None:
+ delta = timedelta(days=90)
+ dttm = datetime(2022, 3, 18, 0, 0, 0)
+ with freeze_time(dttm):
+ cache.set(FIRST_KEY, FIRST_KEY_INITIAL_VALUE, int(delta.total_seconds()))
+ assert cache.get(FIRST_KEY) == FIRST_KEY_INITIAL_VALUE
+ with freeze_time(dttm + delta - timedelta(seconds=1)):
+ assert cache.has(FIRST_KEY)
+ assert cache.get(FIRST_KEY) == FIRST_KEY_INITIAL_VALUE
+ with freeze_time(dttm + delta + timedelta(seconds=1)):
+ assert cache.has(FIRST_KEY) is False
+ assert cache.get(FIRST_KEY) is None
Review comment:
Btw I truly LOVE `freeze_time`! ❤️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830199640
##########
File path: tests/integration_tests/key_value/cache_test.py
##########
@@ -0,0 +1,76 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
Ah, good point! Ignore my comment!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ef1984e) into [master](https://codecov.io/gh/apache/superset/commit/51061f0d672abca29f84943acb16a37403f25c2e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (51061f0) will **decrease** coverage by `0.09%`.
> The diff coverage is `89.58%`.
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
- Coverage 66.76% 66.66% -0.10%
==========================================
Files 1670 1672 +2
Lines 64392 64488 +96
Branches 6499 6501 +2
==========================================
Hits 42991 42991
- Misses 19718 19813 +95
- Partials 1683 1684 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.47% <5.88%> (-0.06%)` | :arrow_down: |
| mysql | `?` | |
| postgres | `?` | |
| presto | `52.32% <5.88%> (-0.06%)` | :arrow_down: |
| python | `82.15% <89.58%> (-0.29%)` | :arrow_down: |
| sqlite | `81.80% <89.58%> (+0.03%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <0.00%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `91.83% <75.00%> (+2.36%)` | :arrow_up: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NhY2hlLnB5) | `100.00% <100.00%> (ø)` | |
| [superset/utils/cache\_manager.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvY2FjaGVfbWFuYWdlci5weQ==) | `100.00% <100.00%> (+6.25%)` | :arrow_up: |
| ... and [47 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [51061f0...ef1984e](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072674182
/testenv up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830113205
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ try:
Review comment:
The update command requires the entry pair to exist. The way I see it, we have two options:
1. check if the entry exists. If so, update, otherwise add
2. always delete and then add
As both require two statements, I think they're mostly equal in terms of performance overhead. I think the only difference is that the `created_on`/`updated_on` will be different.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
michael-s-molina commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830019202
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
Review comment:
Can we use `_prune` instead of `_purge`? If you check Flask-Caching backends, that's the name they are using.
##########
File path: superset-frontend/cypress-base/cypress/integration/dashboard/key_value.test.ts
##########
@@ -27,7 +27,7 @@ interface QueryString {
native_filters_key: string;
}
-xdescribe('nativefiler url param key', () => {
+describe('nativefiler url param key', () => {
Review comment:
I'm not sure this problem is related to the cache. Although it's the `key_value.test` that is failing, you can see that the error is happening in `dashboardHelper/getChartGridComponent`
##########
File path: superset/utils/cache_manager.py
##########
@@ -40,27 +38,27 @@ def _init_cache(
) -> None:
cache_config = app.config[cache_config_key]
cache_type = cache_config.get("CACHE_TYPE")
- if app.debug and cache_type is None:
- cache_threshold = cache_config.get("CACHE_THRESHOLD", math.inf)
+ if required and cache_type in (None, "SupersetCache"):
+ if cache_type is None:
+ logger.warning(
+ "Falling back to built-in key-value cache for the following "
Review comment:
Maybe "Falling back to the built-in cache, that stores data in the metadata database, for the following..."
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ try:
Review comment:
Can't we just use the update command?
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ try:
+ DeleteKeyValueCommand(
+ resource=RESOURCE, key_type=KEY_TYPE, key=self.get_key(key),
+ ).run()
+ except KeyValueCreateFailedError:
+ pass
+ return self.add(key, value, timeout)
+
+ def add(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.create import CreateKeyValueCommand
+
+ try:
+ CreateKeyValueCommand(
+ resource=RESOURCE,
+ value=value,
+ key_type=KEY_TYPE,
+ key=self.get_key(key),
+ expires_on=self.get_expiry(timeout),
+ ).run()
+ self._purge()
+ return True
+ except KeyValueCreateFailedError:
+ return False
+
+ def get(self, key: str) -> Any:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.get import GetKeyValueCommand
+
+ return GetKeyValueCommand(
+ resource=RESOURCE, key_type=KEY_TYPE, key=self.get_key(key),
+ ).run()
+
+ def has(self, key: str) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.get import GetKeyValueCommand
+
+ entry = GetKeyValueCommand(
Review comment:
Maybe reference `get` here so you can keep the logic in just one place?
##########
File path: docs/docs/installation/cache.mdx
##########
@@ -18,10 +18,17 @@ The following cache configurations can be customized:
- Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`.
- Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG`
-Please note, that Dashboard and Explore caching is required. When running Superset in debug mode, both Explore and Dashboard caches will default to `SimpleCache`;
-However, trying to run Superset in non-debug mode without defining a cache for these will cause the application to fail on startup. When running
-superset in single-worker mode, any cache backend is supported. However, when running Superset in on a multi-worker setup, a dedicated cache is required. For this
-we recommend using either Redis or Memcached:
+Please note, that Dashboard and Explore caching is required. If these caches are undefined, Superset falls back to using a built-in cache that stores data
+in the metadata database. While it is recommended to use a dedicated cache, the built-in cache can also be used to cache other data.
+For example, to use the built-in cache to store chart data, use the following config:
+
+```python
+DATA_CACHE_CONFIG = {
+ "CACHE_TYPE": "SupersetCache",
+ "CACHE_KEY_PREFIX": "superset_results", # make sure this string is unique to avoid collisions
+ "CACHE_DEFAULT_TIMEOUT": 86400, # 60 seconds * 60 minutes * 24 hours
Review comment:
Maybe use `int(timedelta(hours=24).total_seconds())`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830194537
##########
File path: tests/integration_tests/key_value/cache_test.py
##########
@@ -0,0 +1,76 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
My thinking here was to make it an integration test to ensure it works on all officially supported backends.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4e9c2f6) into [master](https://codecov.io/gh/apache/superset/commit/51061f0d672abca29f84943acb16a37403f25c2e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (51061f0) will **increase** coverage by `0.05%`.
> The diff coverage is `87.09%`.
> :exclamation: Current head 4e9c2f6 differs from pull request most recent head ef1984e. Consider uploading reports for the commit ef1984e to get more accurate results
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.76% 66.82% +0.05%
==========================================
Files 1670 1672 +2
Lines 64392 64488 +96
Branches 6499 6501 +2
==========================================
+ Hits 42991 43092 +101
+ Misses 19718 19712 -6
- Partials 1683 1684 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.47% <20.43%> (-0.06%)` | :arrow_down: |
| mysql | `82.00% <87.60%> (+0.03%)` | :arrow_up: |
| postgres | `82.05% <87.60%> (+0.03%)` | :arrow_up: |
| presto | `52.32% <20.43%> (-0.06%)` | :arrow_down: |
| python | `82.47% <87.60%> (+0.03%)` | :arrow_up: |
| sqlite | `81.80% <87.60%> (+0.03%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ns/legacy-plugin-chart-horizon/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LWhvcml6b24vc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...acy-plugin-chart-paired-t-test/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhaXJlZC10LXRlc3Qvc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gin-chart-parallel-coordinates/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhcmFsbGVsLWNvb3JkaW5hdGVzL3NyYy9jb250cm9sUGFuZWwudHM=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...legacy-plugin-chart-partition/src/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhcnRpdGlvbi9zcmMvY29udHJvbFBhbmVsLnRzeA==) | `33.33% <ø> (+8.33%)` | :arrow_up: |
| [...egacy-plugin-chart-pivot-table/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBpdm90LXRhYmxlL3NyYy9jb250cm9sUGFuZWwudHM=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gins/legacy-plugin-chart-rose/src/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXJvc2Uvc3JjL2NvbnRyb2xQYW5lbC50c3g=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...ns/legacy-plugin-chart-treemap/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXRyZWVtYXAvc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gacy-preset-chart-nvd3/src/DistBar/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcHJlc2V0LWNoYXJ0LW52ZDMvc3JjL0Rpc3RCYXIvY29udHJvbFBhbmVsLnRz) | `11.11% <ø> (+1.11%)` | :arrow_up: |
| [...chart-echarts/src/Timeseries/Area/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvVGltZXNlcmllcy9BcmVhL2NvbnRyb2xQYW5lbC50c3g=) | `40.00% <ø> (+6.66%)` | :arrow_up: |
| [...charts/src/Timeseries/Regular/Bar/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvVGltZXNlcmllcy9SZWd1bGFyL0Jhci9jb250cm9sUGFuZWwudHN4) | `40.00% <ø> (+6.66%)` | :arrow_up: |
| ... and [30 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [51061f0...ef1984e](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (22f52f6) into [master](https://codecov.io/gh/apache/superset/commit/d645579cdd64b7fe7f9dde4a8da000dd7db51ce9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d645579) will **increase** coverage by `0.03%`.
> The diff coverage is `90.19%`.
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.53% 66.57% +0.03%
==========================================
Files 1667 1670 +3
Lines 64360 64501 +141
Branches 6493 6493
==========================================
+ Hits 42824 42939 +115
- Misses 19854 19880 +26
Partials 1682 1682
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.54% <9.45%> (-0.11%)` | :arrow_down: |
| mysql | `?` | |
| postgres | `81.57% <90.19%> (+0.04%)` | :arrow_up: |
| presto | `52.39% <9.45%> (-0.11%)` | :arrow_down: |
| python | `81.95% <90.19%> (-0.01%)` | :arrow_down: |
| sqlite | `81.35% <90.19%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <66.66%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `93.87% <83.33%> (+4.40%)` | :arrow_up: |
| [superset/key\_value/commands/upsert.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3Vwc2VydC5weQ==) | `89.79% <89.79%> (ø)` | |
| [superset/extensions/metastore\_cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXh0ZW5zaW9ucy9tZXRhc3RvcmVfY2FjaGUucHk=) | `98.18% <98.18%> (ø)` | |
| ... and [12 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d645579...22f52f6](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830113205
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ try:
Review comment:
The update command requires the entry pair to exists. The way I see it, we have two options:
1. check if the entry exists. If so, update, otherwise add
2. always delete and then add
As both require two statements, I think they're mostly equal in terms of performance overhead. I think the only difference is that the `created_on`/`updated_on` will be different.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072719330
I can confirm that the dashboard and explore ephemeral env now work as intended by defaulting to the new cache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830454998
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
Review comment:
Thanks - many good comments there - I'll be opening a follow-up PR to address those!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (58fd5ba) into [master](https://codecov.io/gh/apache/superset/commit/d645579cdd64b7fe7f9dde4a8da000dd7db51ce9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d645579) will **increase** coverage by `0.03%`.
> The diff coverage is `90.13%`.
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.53% 66.57% +0.03%
==========================================
Files 1667 1670 +3
Lines 64360 64500 +140
Branches 6493 6493
==========================================
+ Hits 42824 42942 +118
- Misses 19854 19876 +22
Partials 1682 1682
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.54% <8.21%> (-0.11%)` | :arrow_down: |
| mysql | `?` | |
| postgres | `81.58% <90.13%> (+0.05%)` | :arrow_up: |
| presto | `52.38% <8.21%> (-0.11%)` | :arrow_down: |
| python | `81.96% <90.13%> (+0.01%)` | :arrow_up: |
| sqlite | `81.35% <90.13%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <66.66%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `93.87% <83.33%> (+4.40%)` | :arrow_up: |
| [superset/key\_value/commands/upsert.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3Vwc2VydC5weQ==) | `89.79% <89.79%> (ø)` | |
| [superset/extensions/metastore\_cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXh0ZW5zaW9ucy9tZXRhc3RvcmVfY2FjaGUucHk=) | `98.18% <98.18%> (ø)` | |
| ... and [8 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d645579...58fd5ba](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830166604
##########
File path: superset-frontend/cypress-base/cypress/integration/dashboard/key_value.test.ts
##########
@@ -27,7 +27,7 @@ interface QueryString {
native_filters_key: string;
}
-xdescribe('nativefiler url param key', () => {
+describe('nativefiler url param key', () => {
Review comment:
Ok - I didn't look into this, I just assumed this was the problem. I'll remove this change so we can do a separate PR to fix this properly 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830449728
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _prune() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
Review comment:
Good idea 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (22f52f6) into [master](https://codecov.io/gh/apache/superset/commit/d645579cdd64b7fe7f9dde4a8da000dd7db51ce9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d645579) will **increase** coverage by `0.05%`.
> The diff coverage is `90.19%`.
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.53% 66.59% +0.05%
==========================================
Files 1667 1670 +3
Lines 64360 64501 +141
Branches 6493 6493
==========================================
+ Hits 42824 42952 +128
- Misses 19854 19867 +13
Partials 1682 1682
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.54% <9.45%> (-0.11%)` | :arrow_down: |
| mysql | `81.53% <90.19%> (+0.05%)` | :arrow_up: |
| postgres | `81.57% <90.19%> (+0.04%)` | :arrow_up: |
| presto | `52.39% <9.45%> (-0.11%)` | :arrow_down: |
| python | `81.99% <90.19%> (+0.03%)` | :arrow_up: |
| sqlite | `81.35% <90.19%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <66.66%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `93.87% <83.33%> (+4.40%)` | :arrow_up: |
| [superset/key\_value/commands/upsert.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3Vwc2VydC5weQ==) | `89.79% <89.79%> (ø)` | |
| [superset/extensions/metastore\_cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXh0ZW5zaW9ucy9tZXRhc3RvcmVfY2FjaGUucHk=) | `98.18% <98.18%> (ø)` | |
| ... and [8 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d645579...22f52f6](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (58fd5ba) into [master](https://codecov.io/gh/apache/superset/commit/d645579cdd64b7fe7f9dde4a8da000dd7db51ce9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d645579) will **increase** coverage by `0.05%`.
> The diff coverage is `90.13%`.
> :exclamation: Current head 58fd5ba differs from pull request most recent head 22f52f6. Consider uploading reports for the commit 22f52f6 to get more accurate results
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.53% 66.59% +0.05%
==========================================
Files 1667 1670 +3
Lines 64360 64500 +140
Branches 6493 6493
==========================================
+ Hits 42824 42955 +131
- Misses 19854 19863 +9
Partials 1682 1682
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.54% <8.21%> (-0.11%)` | :arrow_down: |
| mysql | `81.53% <90.13%> (+0.05%)` | :arrow_up: |
| postgres | `81.58% <90.13%> (+0.05%)` | :arrow_up: |
| presto | `52.38% <8.21%> (-0.11%)` | :arrow_down: |
| python | `82.00% <90.13%> (+0.05%)` | :arrow_up: |
| sqlite | `81.35% <90.13%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <66.66%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `93.87% <83.33%> (+4.40%)` | :arrow_up: |
| [superset/key\_value/commands/upsert.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3Vwc2VydC5weQ==) | `89.79% <89.79%> (ø)` | |
| [superset/extensions/metastore\_cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXh0ZW5zaW9ucy9tZXRhc3RvcmVfY2FjaGUucHk=) | `98.18% <98.18%> (ø)` | |
| ... and [4 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d645579...22f52f6](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830198980
##########
File path: docs/docs/installation/cache.mdx
##########
@@ -18,10 +18,17 @@ The following cache configurations can be customized:
- Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`.
- Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG`
-Please note, that Dashboard and Explore caching is required. When running Superset in debug mode, both Explore and Dashboard caches will default to `SimpleCache`;
-However, trying to run Superset in non-debug mode without defining a cache for these will cause the application to fail on startup. When running
-superset in single-worker mode, any cache backend is supported. However, when running Superset in on a multi-worker setup, a dedicated cache is required. For this
-we recommend using either Redis or Memcached:
+Please note, that Dashboard and Explore caching is required. If these caches are undefined, Superset falls back to using a built-in cache that stores data
+in the metadata database. While it is recommended to use a dedicated cache, the built-in cache can also be used to cache other data.
+For example, to use the built-in cache to store chart data, use the following config:
+
+```python
+DATA_CACHE_CONFIG = {
+ "CACHE_TYPE": "SupersetCache",
+ "CACHE_KEY_PREFIX": "superset_results", # make sure this string is unique to avoid collisions
+ "CACHE_DEFAULT_TIMEOUT": 86400, # 60 seconds * 60 minutes * 24 hours
Review comment:
Actually, to avoid having to add an import statement to the example, I think it may be simpler if we leave it as is. But if you think this is more readable I'm happy to change it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830201960
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
Review comment:
I love that commercial so much whenever I need any accessories even remotely related to construction work I buy Blåkläder! 😆
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
michael-s-molina commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830244118
##########
File path: docs/docs/installation/cache.mdx
##########
@@ -18,10 +18,17 @@ The following cache configurations can be customized:
- Dashboard filter state (required): `FILTER_STATE_CACHE_CONFIG`.
- Explore chart form data (required): `EXPLORE_FORM_DATA_CACHE_CONFIG`
-Please note, that Dashboard and Explore caching is required. When running Superset in debug mode, both Explore and Dashboard caches will default to `SimpleCache`;
-However, trying to run Superset in non-debug mode without defining a cache for these will cause the application to fail on startup. When running
-superset in single-worker mode, any cache backend is supported. However, when running Superset in on a multi-worker setup, a dedicated cache is required. For this
-we recommend using either Redis or Memcached:
+Please note, that Dashboard and Explore caching is required. If these caches are undefined, Superset falls back to using a built-in cache that stores data
+in the metadata database. While it is recommended to use a dedicated cache, the built-in cache can also be used to cache other data.
+For example, to use the built-in cache to store chart data, use the following config:
+
+```python
+DATA_CACHE_CONFIG = {
+ "CACHE_TYPE": "SupersetCache",
+ "CACHE_KEY_PREFIX": "superset_results", # make sure this string is unique to avoid collisions
+ "CACHE_DEFAULT_TIMEOUT": 86400, # 60 seconds * 60 minutes * 24 hours
Review comment:
No problem 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830449651
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _prune() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ DeleteKeyValueCommand(
+ resource=RESOURCE, key_type=KEY_TYPE, key=self.get_key(key),
+ ).run()
Review comment:
We already have an `UpdateKeyValueCommand`. To disambiguate, perhaps adding an `UpsertKeyValueCommand` could be more explicit, I'll add that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] michael-s-molina commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
michael-s-molina commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830182983
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
Review comment:
Oh man, this commercial was a good reference! 🤣
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] ktmud commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
ktmud commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830356952
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
Review comment:
Would it make more sense to put this file under `superset/extensions/key_value_cache.py`? The rational is `superset.key_value` as an entity model should not depend on a Flask extension.
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
Review comment:
Linking my previous comment about KeyType in case you missed it: https://github.com/apache/superset/pull/19078#discussion_r828558897
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _prune() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ DeleteKeyValueCommand(
+ resource=RESOURCE, key_type=KEY_TYPE, key=self.get_key(key),
+ ).run()
Review comment:
Instead of `delete` then `add` (two SQL queries), can we implemented a `UpdateKeyValueCommand` command instead?
##########
File path: superset/utils/cache_manager.py
##########
@@ -40,27 +38,27 @@ def _init_cache(
) -> None:
cache_config = app.config[cache_config_key]
cache_type = cache_config.get("CACHE_TYPE")
- if app.debug and cache_type is None:
- cache_threshold = cache_config.get("CACHE_THRESHOLD", math.inf)
+ if required and cache_type in (None, "SupersetCache"):
+ if cache_type is None:
+ logger.warning(
+ "Falling back to the built-in cache, that stores data in the "
Review comment:
Are we sure we want to show this warning when users specifically set cache type to `SupersetCache`? It's not "falling back" at least.
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _prune() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
Review comment:
```suggestion
def _get_expiry(self, timeout: Optional[int]) -> datetime:
```
Maybe this should be a private function?
Flask-Caching's `SimpleCache` also called this function [BaseCache._normalize_timeout](https://github.com/pallets-eco/flask-caching/blob/e3440f3601061e3732f920a59f31d154c11d73ba/src/flask_caching/backends/simplecache.py#L71), maybe we should follow the same convention? (It also ignores negative timeout)
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
Review comment:
```suggestion
class SupersetMetastoreCache(BaseCache):
```
Not sure if it's worth the verboseness, but would `SupersetKeyValueCache` or `SupersetMetastoreCache` be a better name because `SupersetCache` sounds a little too generic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (22f52f6) into [master](https://codecov.io/gh/apache/superset/commit/d645579cdd64b7fe7f9dde4a8da000dd7db51ce9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d645579) will **decrease** coverage by `0.09%`.
> The diff coverage is `90.19%`.
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
- Coverage 66.53% 66.44% -0.10%
==========================================
Files 1667 1670 +3
Lines 64360 64501 +141
Branches 6493 6493
==========================================
+ Hits 42824 42859 +35
- Misses 19854 19960 +106
Partials 1682 1682
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.54% <9.45%> (-0.11%)` | :arrow_down: |
| mysql | `?` | |
| postgres | `?` | |
| presto | `52.39% <9.45%> (-0.11%)` | :arrow_down: |
| python | `81.70% <90.19%> (-0.26%)` | :arrow_down: |
| sqlite | `81.35% <90.19%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/dashboards/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `92.59% <ø> (ø)` | |
| [superset/dashboards/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGFzaGJvYXJkcy9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `82.35% <ø> (ø)` | |
| [superset/explore/permalink/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvY3JlYXRlLnB5) | `93.33% <ø> (ø)` | |
| [superset/explore/permalink/commands/get.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXhwbG9yZS9wZXJtYWxpbmsvY29tbWFuZHMvZ2V0LnB5) | `88.57% <ø> (ø)` | |
| [superset/key\_value/commands/delete.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZS5weQ==) | `86.11% <50.00%> (-2.78%)` | :arrow_down: |
| [superset/key\_value/commands/update.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3VwZGF0ZS5weQ==) | `89.58% <66.66%> (-1.91%)` | :arrow_down: |
| [superset/key\_value/commands/delete\_expired.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2RlbGV0ZV9leHBpcmVkLnB5) | `80.00% <80.00%> (ø)` | |
| [superset/key\_value/commands/create.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL2NyZWF0ZS5weQ==) | `93.87% <83.33%> (+4.40%)` | :arrow_up: |
| [superset/key\_value/commands/upsert.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQva2V5X3ZhbHVlL2NvbW1hbmRzL3Vwc2VydC5weQ==) | `89.79% <89.79%> (ø)` | |
| [superset/extensions/metastore\_cache.py](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZXh0ZW5zaW9ucy9tZXRhc3RvcmVfY2FjaGUucHk=) | `98.18% <98.18%> (ø)` | |
| ... and [23 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d645579...22f52f6](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830189845
##########
File path: tests/integration_tests/key_value/cache_test.py
##########
@@ -0,0 +1,76 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
This can go in the `unit_tests` directory instead.
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,113 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
Review comment:
It looks like `namespace` is optional (based on line 36)?
```suggestion
def __init__(self, namespace: Optional[UUID], default_timeout: int = 300) -> None:
super().__init__(default_timeout)
self.namespace = namespace or uuid4()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro merged pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
villebro merged pull request #19232:
URL: https://github.com/apache/superset/pull/19232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830141768
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete_expired import (
+ DeleteExpiredKeyValueCommand,
+ )
+
+ DeleteExpiredKeyValueCommand(resource=RESOURCE).run()
+
+ def get_expiry(self, timeout: Optional[int]) -> datetime:
+ return datetime.now() + timedelta(seconds=timeout or self.default_timeout)
+
+ def set(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.delete import DeleteKeyValueCommand
+
+ try:
+ DeleteKeyValueCommand(
+ resource=RESOURCE, key_type=KEY_TYPE, key=self.get_key(key),
+ ).run()
+ except KeyValueCreateFailedError:
+ pass
+ return self.add(key, value, timeout)
+
+ def add(self, key: str, value: Any, timeout: Optional[int] = None) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.create import CreateKeyValueCommand
+
+ try:
+ CreateKeyValueCommand(
+ resource=RESOURCE,
+ value=value,
+ key_type=KEY_TYPE,
+ key=self.get_key(key),
+ expires_on=self.get_expiry(timeout),
+ ).run()
+ self._purge()
+ return True
+ except KeyValueCreateFailedError:
+ return False
+
+ def get(self, key: str) -> Any:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.get import GetKeyValueCommand
+
+ return GetKeyValueCommand(
+ resource=RESOURCE, key_type=KEY_TYPE, key=self.get_key(key),
+ ).run()
+
+ def has(self, key: str) -> bool:
+ # pylint: disable=import-outside-toplevel
+ from superset.key_value.commands.get import GetKeyValueCommand
+
+ entry = GetKeyValueCommand(
Review comment:
Oops!
##########
File path: superset/utils/cache_manager.py
##########
@@ -40,27 +38,27 @@ def _init_cache(
) -> None:
cache_config = app.config[cache_config_key]
cache_type = cache_config.get("CACHE_TYPE")
- if app.debug and cache_type is None:
- cache_threshold = cache_config.get("CACHE_THRESHOLD", math.inf)
+ if required and cache_type in (None, "SupersetCache"):
+ if cache_type is None:
+ logger.warning(
+ "Falling back to built-in key-value cache for the following "
Review comment:
I like that 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072639340
# [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19232](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4e9c2f6) into [master](https://codecov.io/gh/apache/superset/commit/51061f0d672abca29f84943acb16a37403f25c2e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (51061f0) will **increase** coverage by `0.05%`.
> The diff coverage is `87.09%`.
> :exclamation: Current head 4e9c2f6 differs from pull request most recent head 78dfe29. Consider uploading reports for the commit 78dfe29 to get more accurate results
```diff
@@ Coverage Diff @@
## master #19232 +/- ##
==========================================
+ Coverage 66.76% 66.82% +0.05%
==========================================
Files 1670 1672 +2
Lines 64392 64488 +96
Branches 6499 6501 +2
==========================================
+ Hits 42991 43092 +101
+ Misses 19718 19712 -6
- Partials 1683 1684 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `52.47% <20.43%> (-0.06%)` | :arrow_down: |
| mysql | `82.00% <87.60%> (+0.03%)` | :arrow_up: |
| postgres | `82.05% <87.60%> (+0.03%)` | :arrow_up: |
| presto | `52.32% <20.43%> (-0.06%)` | :arrow_down: |
| python | `82.47% <87.60%> (+0.03%)` | :arrow_up: |
| sqlite | `81.80% <87.60%> (+0.03%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ns/legacy-plugin-chart-horizon/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LWhvcml6b24vc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...acy-plugin-chart-paired-t-test/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhaXJlZC10LXRlc3Qvc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gin-chart-parallel-coordinates/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhcmFsbGVsLWNvb3JkaW5hdGVzL3NyYy9jb250cm9sUGFuZWwudHM=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...legacy-plugin-chart-partition/src/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBhcnRpdGlvbi9zcmMvY29udHJvbFBhbmVsLnRzeA==) | `33.33% <ø> (+8.33%)` | :arrow_up: |
| [...egacy-plugin-chart-pivot-table/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXBpdm90LXRhYmxlL3NyYy9jb250cm9sUGFuZWwudHM=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gins/legacy-plugin-chart-rose/src/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXJvc2Uvc3JjL2NvbnRyb2xQYW5lbC50c3g=) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...ns/legacy-plugin-chart-treemap/src/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LXRyZWVtYXAvc3JjL2NvbnRyb2xQYW5lbC50cw==) | `100.00% <ø> (+50.00%)` | :arrow_up: |
| [...gacy-preset-chart-nvd3/src/DistBar/controlPanel.ts](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcHJlc2V0LWNoYXJ0LW52ZDMvc3JjL0Rpc3RCYXIvY29udHJvbFBhbmVsLnRz) | `11.11% <ø> (+1.11%)` | :arrow_up: |
| [...chart-echarts/src/Timeseries/Area/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvVGltZXNlcmllcy9BcmVhL2NvbnRyb2xQYW5lbC50c3g=) | `40.00% <ø> (+6.66%)` | :arrow_up: |
| [...charts/src/Timeseries/Regular/Bar/controlPanel.tsx](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9wbHVnaW4tY2hhcnQtZWNoYXJ0cy9zcmMvVGltZXNlcmllcy9SZWd1bGFyL0Jhci9jb250cm9sUGFuZWwudHN4) | `40.00% <ø> (+6.66%)` | :arrow_up: |
| ... and [30 more](https://codecov.io/gh/apache/superset/pull/19232/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [51061f0...78dfe29](https://codecov.io/gh/apache/superset/pull/19232?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] github-actions[bot] commented on pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072676014
@villebro Ephemeral environment spinning up at http://35.87.132.23:8080. Credentials are `admin`/`admin`. Please allow several minutes for bootstrapping and startup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1072987205
@ktmud I believe I've addressed all your comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830197391
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,113 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
Review comment:
Good catch! I went back and forth here a few times and that was a leftover from a previous iteration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] github-actions[bot] commented on pull request #19232: feat(key-value): add superset metastore cache
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #19232:
URL: https://github.com/apache/superset/pull/19232#issuecomment-1074223064
Ephemeral environment shutdown and build artifacts deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830165558
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3, uuid4
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
+ def __init__(self, namespace: UUID, default_timeout: int = 300) -> None:
+ super().__init__(default_timeout)
+ self.namespace = namespace or uuid4()
+
+ @classmethod
+ def factory(
+ cls, app: Flask, config: Dict[str, Any], args: List[Any], kwargs: Dict[str, Any]
+ ) -> BaseCache:
+ # base namespace for generating deterministic UUIDs
+ md5_obj = md5()
+ seed = config.get("CACHE_KEY_PREFIX", "")
+ md5_obj.update(seed.encode("utf-8"))
+ kwargs["namespace"] = UUID(md5_obj.hexdigest())
+ return cls(*args, **kwargs)
+
+ def get_key(self, key: str) -> str:
+ return str(uuid3(self.namespace, key))
+
+ @staticmethod
+ def _purge() -> None:
Review comment:
Lol, that's what I tried to call it but ended up misremembering the name of the method 😄 This reminds me of this awesome commercial we had on TV here in the Nordics (I hope it's viewable elsewhere; subtitles in Finnish): https://www.youtube.com/watch?v=jkqVaHCzZaw
"That's what I said, `_prune`!" 👷
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830449933
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
+
+from superset.key_value.exceptions import KeyValueCreateFailedError
+from superset.key_value.types import KeyType
+
+RESOURCE = "superset_cache"
+KEY_TYPE: KeyType = "uuid"
+
+
+class SupersetCache(BaseCache):
Review comment:
I like `SupersetMetastoreCache`, let's go with that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830449791
##########
File path: superset/key_value/cache.py
##########
@@ -0,0 +1,110 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from datetime import datetime, timedelta
+from hashlib import md5
+from typing import Any, Dict, List, Optional
+from uuid import UUID, uuid3
+
+from flask import Flask
+from flask_caching import BaseCache
Review comment:
Good idea 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #19232: feat(key-value): add superset cache
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #19232:
URL: https://github.com/apache/superset/pull/19232#discussion_r830449897
##########
File path: superset/utils/cache_manager.py
##########
@@ -40,27 +38,27 @@ def _init_cache(
) -> None:
cache_config = app.config[cache_config_key]
cache_type = cache_config.get("CACHE_TYPE")
- if app.debug and cache_type is None:
- cache_threshold = cache_config.get("CACHE_THRESHOLD", math.inf)
+ if required and cache_type in (None, "SupersetCache"):
+ if cache_type is None:
+ logger.warning(
+ "Falling back to the built-in cache, that stores data in the "
Review comment:
We're not raising the warning unless `cache_type is None` (see a few lines above)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org