You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "eschutho (via GitHub)" <gi...@apache.org> on 2025/11/15 02:48:11 UTC
[PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
eschutho opened a new pull request, #36126:
URL: https://github.com/apache/superset/pull/36126
Previously, the cache was being saved in two problematic scenarios:
1. When entering COMPUTING state (line 280) - before image generation
2. Always after processing (line 302) - even when image generation failed
This caused issues where the /screenshot endpoint would find a cache key but without an actual image, resulting in 404 errors.
Changes:
- Removed cache.set() call after entering COMPUTING state
- Moved cache.set() and status logging inside the `if image:` block
- Cache is now only saved when image generation and processing succeed
This ensures that cache entries always contain valid image data.
<!---
Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/
Example:
fix(dashboard): load charts correctly
-->
### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #36126:
URL: https://github.com/apache/superset/pull/36126#discussion_r2548563067
##########
superset/utils/screenshots.py:
##########
@@ -43,23 +68,141 @@
from flask_caching import Cache
+class StatusValues(Enum):
+ PENDING = "Pending"
+ COMPUTING = "Computing"
+ UPDATED = "Updated"
+ ERROR = "Error"
+
+
+class ScreenshotCachePayloadType(TypedDict):
+ image: str | None
+ timestamp: str
+ status: str
+
+
+class ScreenshotCachePayload:
+ def __init__(
+ self,
+ image: bytes | None = None,
+ status: StatusValues = StatusValues.PENDING,
+ timestamp: str = "",
+ ):
+ self._image = image
+ self._timestamp = timestamp or datetime.now().isoformat()
Review Comment:
this suggestion is not related to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
Posted by "dpgaspar (via GitHub)" <gi...@apache.org>.
dpgaspar commented on PR #36126:
URL: https://github.com/apache/superset/pull/36126#issuecomment-3547858915
LGTM, but tests need to be fixed and make sure moving the cache.set inside the `if image` won't break anything
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho commented on code in PR #36126:
URL: https://github.com/apache/superset/pull/36126#discussion_r2548563405
##########
superset/utils/screenshots.py:
##########
@@ -16,23 +16,48 @@
# under the License.
from __future__ import annotations
+import base64
import logging
+from datetime import datetime
+from enum import Enum
from io import BytesIO
-from typing import TYPE_CHECKING
+from typing import cast, TYPE_CHECKING, TypedDict
-from flask import current_app
+from flask import current_app as app
+from superset import feature_flag_manager, thumbnail_cache
+from superset.exceptions import ScreenshotImageNotAvailableException
+from superset.extensions import event_logger
from superset.utils.hashing import md5_sha_from_dict
from superset.utils.urls import modify_url_query
from superset.utils.webdriver import (
ChartStandaloneMode,
DashboardStandaloneMode,
- WebDriverProxy,
+ WebDriver,
+ WebDriverPlaywright,
+ WebDriverSelenium,
WindowSize,
)
logger = logging.getLogger(__name__)
+# Import Playwright availability and install message
+try:
+ from superset.utils.webdriver import (
+ PLAYWRIGHT_AVAILABLE,
+ PLAYWRIGHT_INSTALL_MESSAGE,
+ )
+except ImportError:
+ PLAYWRIGHT_AVAILABLE = False
+ PLAYWRIGHT_INSTALL_MESSAGE = "Playwright module not found"
Review Comment:
This suggestion is not related to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #36126:
URL: https://github.com/apache/superset/pull/36126#discussion_r2529486076
##########
superset/utils/screenshots.py:
##########
@@ -75,67 +218,42 @@ def cache_key(
}
return md5_sha_from_dict(args)
- def get_screenshot(
- self, user: User, window_size: WindowSize | None = None
- ) -> bytes | None:
- driver = self.driver(window_size)
- self.screenshot = driver.get_screenshot(self.url, self.element, user)
- return self.screenshot
-
- def get(
- self,
- user: User = None,
- cache: Cache = None,
- thumb_size: WindowSize | None = None,
- ) -> BytesIO | None:
- """
- Get thumbnail screenshot has BytesIO from cache or fetch
-
- :param user: None to use current user or User Model to login and fetch
- :param cache: The cache to use
- :param thumb_size: Override thumbnail site
- """
- payload: bytes | None = None
- cache_key = self.cache_key(self.window_size, thumb_size)
- if cache:
- payload = cache.get(cache_key)
- if not payload:
- payload = self.compute_and_cache(
- user=user, thumb_size=thumb_size, cache=cache
- )
- else:
- logger.info("Loaded thumbnail from cache: %s", cache_key)
- if payload:
- return BytesIO(payload)
- return None
-
def get_from_cache(
self,
- cache: Cache,
window_size: WindowSize | None = None,
thumb_size: WindowSize | None = None,
- ) -> BytesIO | None:
- cache_key = self.cache_key(window_size, thumb_size)
- return self.get_from_cache_key(cache, cache_key)
+ ) -> ScreenshotCachePayload | None:
+ cache_key = self.get_cache_key(window_size, thumb_size)
+ return self.get_from_cache_key(cache_key)
- @staticmethod
- def get_from_cache_key(cache: Cache, cache_key: str) -> BytesIO | None:
+ @classmethod
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>API compatibility broken by method signature change</b></div>
<div id="fix">
Changing get_from_cache_key from a staticmethod taking (cache, cache_key) to a classmethod taking only (cache_key) breaks the existing call in superset/charts/api.py that passes thumbnail_cache and digest as arguments. This will cause a TypeError at runtime when the chart thumbnail endpoint is accessed. The fix updates the call to pass only the digest and handles the new ScreenshotCachePayload return type by extracting the BytesIO image.
</div>
<details>
<summary>
<b>Code suggestion</b>
</summary>
<blockquote>Check the AI-generated fix before applying</blockquote>
<div id="code">
```
- if img := ChartScreenshot.get_from_cache_key(thumbnail_cache, digest):
- return Response(
- FileWrapper(img), mimetype="image/png", direct_passthrough=True
+ if img := ChartScreenshot.get_from_cache_key(digest):
+ img = img.get_image()
+ return Response(
+ FileWrapper(img), mimetype="image/png", direct_passthrough=True
```
</div>
</details>
</div>
<small><i>Code Review Run <a href=https://github.com/apache/superset/pull/36126#issuecomment-3535451499>#cf50c2</a></i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
##########
superset/utils/screenshots.py:
##########
@@ -16,23 +16,48 @@
# under the License.
from __future__ import annotations
+import base64
import logging
+from datetime import datetime
+from enum import Enum
from io import BytesIO
-from typing import TYPE_CHECKING
+from typing import cast, TYPE_CHECKING, TypedDict
-from flask import current_app
+from flask import current_app as app
+from superset import feature_flag_manager, thumbnail_cache
+from superset.exceptions import ScreenshotImageNotAvailableException
+from superset.extensions import event_logger
from superset.utils.hashing import md5_sha_from_dict
from superset.utils.urls import modify_url_query
from superset.utils.webdriver import (
ChartStandaloneMode,
DashboardStandaloneMode,
- WebDriverProxy,
+ WebDriver,
+ WebDriverPlaywright,
+ WebDriverSelenium,
WindowSize,
)
logger = logging.getLogger(__name__)
+# Import Playwright availability and install message
+try:
+ from superset.utils.webdriver import (
+ PLAYWRIGHT_AVAILABLE,
+ PLAYWRIGHT_INSTALL_MESSAGE,
+ )
+except ImportError:
+ PLAYWRIGHT_AVAILABLE = False
+ PLAYWRIGHT_INSTALL_MESSAGE = "Playwright module not found"
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Missing Playwright constants from webdriver module</b></div>
<div id="fix">
The constants `PLAYWRIGHT_AVAILABLE` and `PLAYWRIGHT_INSTALL_MESSAGE` are imported from `superset.utils.webdriver` but these attributes don't exist in the module. The code has fallback handling in the except block, but the import will still fail. Consider moving the import inside the try block.
</div>
<details>
<summary>
<b>Code suggestion</b>
</summary>
<blockquote>Check the AI-generated fix before applying</blockquote>
<div id="code">
````suggestion
# Import Playwright availability and install message
try:
from superset.utils.webdriver import (
PLAYWRIGHT_AVAILABLE,
PLAYWRIGHT_INSTALL_MESSAGE,
)
except (ImportError, AttributeError):
PLAYWRIGHT_AVAILABLE = False
PLAYWRIGHT_INSTALL_MESSAGE = "Playwright module not found"
````
</div>
</details>
</div>
<small><i>Code Review Run <a href=https://github.com/apache/superset/pull/36126#issuecomment-3535451499>#cf50c2</a></i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
##########
superset/utils/screenshots.py:
##########
@@ -43,23 +68,141 @@
from flask_caching import Cache
+class StatusValues(Enum):
+ PENDING = "Pending"
+ COMPUTING = "Computing"
+ UPDATED = "Updated"
+ ERROR = "Error"
+
+
+class ScreenshotCachePayloadType(TypedDict):
+ image: str | None
+ timestamp: str
+ status: str
+
+
+class ScreenshotCachePayload:
+ def __init__(
+ self,
+ image: bytes | None = None,
+ status: StatusValues = StatusValues.PENDING,
+ timestamp: str = "",
+ ):
+ self._image = image
+ self._timestamp = timestamp or datetime.now().isoformat()
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Datetime now called without timezone</b></div>
<div id="fix">
Multiple `datetime.now()` calls without timezone argument on lines 92, 113, and 150. Consider using `datetime.now(timezone.utc)` for timezone-aware timestamps.
</div>
<details>
<summary>
<b>Code suggestion</b>
</summary>
<blockquote>Check the AI-generated fix before applying</blockquote>
<div id="code">
````suggestion
self._image = image
self._timestamp = timestamp or datetime.now(timezone.utc).isoformat()
````
</div>
</details>
</div>
<small><i>Code Review Run <a href=https://github.com/apache/superset/pull/36126#issuecomment-3535451499>#cf50c2</a></i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #36126:
URL: https://github.com/apache/superset/pull/36126#issuecomment-3535490012
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/36126?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `2 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 67.80%. Comparing base ([`c2baba5`](https://app.codecov.io/gh/apache/superset/commit/c2baba50f9c266f7ead757d7297be5ac5ceb4d99?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`aace7c5`](https://app.codecov.io/gh/apache/superset/commit/aace7c5fcaff2040d5a82e71de65c6a64528e9ae?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/36126?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/utils/screenshots.py](https://app.codecov.io/gh/apache/superset/pull/36126?src=pr&el=tree&filepath=superset%2Futils%2Fscreenshots.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvdXRpbHMvc2NyZWVuc2hvdHMucHk=) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/36126?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #36126 +/- ##
===========================================
+ Coverage 0 67.80% +67.80%
===========================================
Files 0 629 +629
Lines 0 46039 +46039
Branches 0 4989 +4989
===========================================
+ Hits 0 31218 +31218
- Misses 0 13561 +13561
- Partials 0 1260 +1260
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/36126/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/36126/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.85% <0.00%> (?)` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/36126/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `67.44% <0.00%> (?)` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/36126/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `47.48% <0.00%> (?)` | |
| [python](https://app.codecov.io/gh/apache/superset/pull/36126/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `67.77% <0.00%> (?)` | |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/36126/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `67.06% <0.00%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/superset/pull/36126?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #36126:
URL: https://github.com/apache/superset/pull/36126#issuecomment-3535451499
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on PR #36126:
URL: https://github.com/apache/superset/pull/36126#issuecomment-3543327455
Superset uses Git pre-commit hooks courtesy of [pre-commit](https://pre-commit.com/). To install run the following:
```
pip3 install -r requirements/development.txt
pre-commit install
```
A series of checks will now run when you make a git commit.
Alternatively it is possible to run pre-commit by running pre-commit manually:
```
pre-commit run --all-files
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(screenshots): Only cache thumbnails when image generation succeeds [superset]
Posted by "eschutho (via GitHub)" <gi...@apache.org>.
eschutho merged PR #36126:
URL: https://github.com/apache/superset/pull/36126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org