You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "Jkraft7 (via GitHub)" <gi...@apache.org> on 2023/09/08 19:20:44 UTC

[GitHub] [superset] Jkraft7 opened a new issue, #25237: Dashboard without a chart fails when generating thumbnail at screenshot step by celery worker

Jkraft7 opened a new issue, #25237:
URL: https://github.com/apache/superset/issues/25237

   In our superset instance we use celery workers to get thumbnails. We run into an issue where if there is an empty/blank dashboard that exists. It causes selenium errors to continually occur whenever the thumbnail is attempted to be created in the celery worker which leads to massive cpu usage and blockers for other things (such as reports/alerts)
   
   It states selenium is timing out while trying to run the get_screenshot function.
   the error itself is:
   `
   Selenium timed out requesting url http://localhost:8080/superset/dashboard/260/?standalone=3
   Traceback (most recent call last):                                                                                                                                                                                                                                          │
   │   File "/app/superset/utils/webdriver.py", line 187, in get_screenshot                                                                                                                                                                                                      │
   │     WebDriverWait(driver, self._screenshot_load_wait).until(                                                                                                                                                                                                            │
   │   File "/usr/local/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 80, in until                                                                                                                                                                   │
   │     raise TimeoutException(message, screen, stacktrace)                                                                                                                                                                                                                                         │
   │ selenium.common.exceptions.TimeoutException: Message:   
   
   ` 
   This error began occurring after the upgrade from 2.0.0 to 2.1.0. This can be detrimental to a celery worker as if there are not enough threads plus the users generate multiple blank/empty dashboards this can block all other activity in the celery worker.
   
   From my perspective it is exactly related to this: https://github.com/apache/superset/issues/17245
   except this is when we try to generate a thumbnail and not a report. These two functionalities should be running roughly the same logic i believe.
   
   #### How to reproduce the bug
   
   1. Ensure you have thumbnails enabled and are using a celery worker to generate them
   2. Create a dashboard with no charts
   3. Navigate to the Dashboards page
   4. Then check celery/k9s logs of it attempting to generate the thumbnail and you will see this error above occurring for the dashboards without the charts.
   
   ### Expected results
   
   Expect that the dashboard will just generate a blank screenshot or have some form of a default method to handle empty dashboards instead of throwing an error consistently.
   
   ### Actual results
   
   Error is thrown every time selenium attempts to screenshot the dashboard for the thumbnail
   
   #### Screenshots
   
   ![Screenshot 2023-09-05 at 4 25 39 PM](https://github.com/apache/superset/assets/28743961/ee6de6af-7fc6-4b6a-a2b4-3e51afaec248)
   
   
   ### Environment
   
   (please complete the following information):
   
   - browser type and version: Chrome Driver/ Chrome 
   - superset version: `2.1.0`
   - python version: `3.8.16-slim`
   - node.js version: `16-slim`
   - any feature flags active:
   FEATURE_FLAGS = {
           "CLIENT_CACHE": False,
           "ENABLE_EXPLORE_JSON_CSRF_PROTECTION": False,
           "SQLLAB_BACKEND_PERSISTENCE": True,
           "THUMBNAILS": True,
           "THUMBNAILS_SQLA_LISTENERS": True,
           "LISTVIEWS_DEFAULT_CARD_VIEW": True,
           "DASHBOARD_RBAC": True,
           "DASHBOARD_NATIVE_FILTERS": True,
           "ENABLE_TEMPLATE_PROCESSING": True,
           "ALERT_REPORTS": True,
           "DASHBOARD_CROSS_FILTERS": True,
           # Allow for javascript controls components
           # this enables programmers to customize certain charts (like the
           # geospatial ones) by inputing javascript in controls. This exposes
           # an XSS security vulnerability
           "ENABLE_JAVASCRIPT_CONTROLS": False,
           "DRILL_TO_DETAIL": True,
           "GENERIC_CHART_AXES": True,
           "HORIZONTAL_FILTER_BAR": True,
       }
   ### Checklist
   
   Make sure to follow these steps before submitting your issue - thank you!
   
   - [ ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
   - [ ] I have reproduced the issue with at least the latest released version of superset.
   - [ ] I have checked the issue tracker for the same issue and I haven't found one similar.
   
   ### Additional context
   
   Add any other context about the problem here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dashboard without a chart fails when generating thumbnail at screenshot step by celery worker [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #25237: Dashboard without a chart fails when generating thumbnail at screenshot step by celery worker
URL: https://github.com/apache/superset/issues/25237


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dashboard without a chart fails when generating thumbnail at screenshot step by celery worker [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #25237:
URL: https://github.com/apache/superset/issues/25237#issuecomment-2704651663

   Closing this as stale, since it's been quiet for upward of a year. Again, if this is relevant in current versions (4.x, or the 5.0 RCs at the moment) then we can certainly reopen it! :D 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dashboard without a chart fails when generating thumbnail at screenshot step by celery worker [superset]

Posted by "zhaorui2022 (via GitHub)" <gi...@apache.org>.
zhaorui2022 commented on issue #25237:
URL: https://github.com/apache/superset/issues/25237#issuecomment-1751450277

   We also noticed the same issue while testing 3.0.0 version.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dashboard without a chart fails when generating thumbnail at screenshot step by celery worker [superset]

Posted by "mirkomarchiori (via GitHub)" <gi...@apache.org>.
mirkomarchiori commented on issue #25237:
URL: https://github.com/apache/superset/issues/25237#issuecomment-3355672287

   > [@rusackas](https://github.com/rusackas) This is definitely still happening for version 5.0.0rc2 with Playwright
   > 
   > ```
   > Call log:
   >   - waiting for locator(".standalone") to be visible
   > [2025-05-15 09:06:50,579: ERROR/ForkPoolWorker-8] Timed out waiting for chart containers to draw at url http://my-superset:8088/superset/dashboard/227/?standalone=3
   > Traceback (most recent call last):
   >   File "/app/superset/utils/webdriver.py", line 194, in get_screenshot
   >     slice_container_locator.first.wait_for()
   >   File "/app/.venv/lib/python3.11/site-packages/playwright/sync_api/_generated.py", line 17920, in wait_for
   >     self._sync(self._impl_obj.wait_for(timeout=timeout, state=state))
   >   File "/app/.venv/lib/python3.11/site-packages/playwright/_impl/_sync_base.py", line 115, in _sync
   >     return task.result()
   >            ^^^^^^^^^^^^^
   >   File "/app/.venv/lib/python3.11/site-packages/playwright/_impl/_locator.py", line 685, in wait_for
   >     await self._frame.wait_for_selector(
   >   File "/app/.venv/lib/python3.11/site-packages/playwright/_impl/_frame.py", line 323, in wait_for_selector
   >     await self._channel.send("waitForSelector", locals_to_params(locals()))
   >   File "/app/.venv/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 61, in send
   >     return await self._connection.wrap_api_call(
   >            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   >   File "/app/.venv/lib/python3.11/site-packages/playwright/_impl/_connection.py", line 528, in wrap_api_call
   >     raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None
   > playwright._impl._errors.TimeoutError: Locator.wait_for: Timeout 30000ms exceeded.
   > ```
   
   Where you able to fix this? I am getting the same error on superset helm chart 0.14.3 (custom superset 4.1.2 image with playwright enabled) while trying to generate a report for a dahsboard with multiple charts where most of them display nothing because of an "Unexpected error" (unable to get data from the table)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dashboard without a chart fails when generating thumbnail at screenshot step by celery worker [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #25237:
URL: https://github.com/apache/superset/issues/25237#issuecomment-1998001195

   Thanks for confirming this was happening in 3.0.0. Is it still in 3.1.x or 3.0.x or 4.0? I assume so, but it's been quite a while since nay updates happened here, so I want to make sure this still valid before it becomes at risk of being closed as stale. ALso have you tried the `PLAYWRIGHT_REPORTS_AND_THUMBNAILS` flag?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Dashboard without a chart fails when generating thumbnail at screenshot step by celery worker [superset]

Posted by "amolsr (via GitHub)" <gi...@apache.org>.
amolsr commented on issue #25237:
URL: https://github.com/apache/superset/issues/25237#issuecomment-2048967516

   Hi, we are facing similar issue. caching the dashboard before the report is generated should help in these type of scenario. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org