You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "leagueofsoups (via GitHub)" <gi...@apache.org> on 2023/03/31 17:07:07 UTC

[GitHub] [superset] leagueofsoups opened a new issue, #23546: superset 2.0.1 dashboard and charts async update

leagueofsoups opened a new issue, #23546:
URL: https://github.com/apache/superset/issues/23546

   I can't celery workers for update dashboard and charts
   
   #### How to reproduce the bug
   
   1. Use this configuration fragment
   ```
   class CeleryConfig(object):
       broker_url = "redis://<fix_it>/0"
       imports = (
         'superset.sql_lab',
         'superset.tasks',
         'superset.tasks.thumbnails',
         'superset.charts',
         'superset.dashboards'
       )
       result_backend = "redis://<fix_it>/0"
       worker_log_level = 'DEBUG'
       worker_prefetch_multiplier = 10
       task_acks_late = True
   
       task_annotations = {
           "tasks.add": {"rate_limit": "10/s"},
           "sql_lab.get_sql_results": {"rate_limit": "100/s"},
           "email_reports.send": {
               "rate_limit": "1/s",
               "time_limit": 120,
               "soft_time_limit": 150,
               "ignore_result": True,
           },
       }
       beat_schedule = {
           "email_reports.schedule_hourly": {
               "task": "email_reports.schedule_hourly",
               "schedule": crontab(minute=1, hour="*"),
           },
   
       }
   CELERY_CONFIG = CeleryConfig
   ```
   2. Add async parametres
   ```
   GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
       "port": 6379,
       "host": "<FIXIT>",
       "password": "<FIXIT>",
       "db": 3,
       "ssl": False,
   }
   GLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX = "async-events-"
   GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT = 1000
   GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT_FIREHOSE = 1000000
   GLOBAL_ASYNC_QUERIES_JWT_COOKIE_NAME = "async-token"
   GLOBAL_ASYNC_QUERIES_JWT_COOKIE_SECURE = False
   GLOBAL_ASYNC_QUERIES_JWT_COOKIE_DOMAIN = None
   GLOBAL_ASYNC_QUERIES_JWT_SECRET =  "<FIXIT>" 
   GLOBAL_ASYNC_QUERIES_TRANSPORT = "polling"
   GLOBAL_ASYNC_QUERIES_POLLING_DELAY = int(
       timedelta(milliseconds=500).total_seconds() * 1000
   )
   ```
   3. add flag
   ```
   FEATURE_FLAGS = {
   'GLOBAL_ASYNC_QUERIES': True,
   ...
   ```
   4. enable Advanced > Performance > Asynchronous query execution for datasource
   5. open dashboards/chard and use this datasource (force refrash)
   
   ### Expected results
   
   Found async tasks in celery
   
   ### Actual results
   
   Have only this tasks names:
   load_chart_data_into_cache
   sql_lab.get_sql_results
   load_explore_json_into_cache
   
   ### Environment
   
   - browser type and version: any
   - superset version: `2.0.1`
   - python version: `Python 3.8.13`
   - node.js version: `without node js`
   - any feature flags active: GLOBAL_ASYNC_QUERIES + ENABLE_TEMPLATE_PROCESSING
   
   ### Checklist
   
   - [ ] I Have async tasks
   
   ### Additional context
   
   pip freeze
   ```
   aiohttp==3.8.3
   aiosignal==1.3.1
   alembic==1.9.1
   amqp==5.1.1
   apache-superset==2.0.1
   apispec==3.3.2
   asttokens==2.2.1
   async-generator==1.10
   async-timeout==4.0.2
   attrs==22.2.0
   Authlib==1.2.0
   Babel==2.11.0
   backcall==0.2.0
   backoff==2.2.1
   billiard==3.6.4.0
   bitarray==2.6.1
   bleach==3.3.1
   boto3==1.26.39
   botocore==1.29.39
   Brotli==1.0.9
   cachelib==0.4.1
   celery==5.2.2
   certifi==2022.12.7
   cffi==1.15.1
   charset-normalizer==2.0.12
   click==8.1.3
   click-didyoumean==0.3.0
   click-plugins==1.1.1
   click-repl==0.2.0
   colorama==0.4.6
   convertdate==2.4.0
   cron-descriptor==1.2.32
   croniter==1.3.8
   cryptography==38.0.2
   decorator==5.1.1
   defusedxml==0.7.1
   deprecation==2.1.0
   dnspython==2.2.1
   email-validator==1.3.0
   exceptiongroup==1.1.0
   executing==1.2.0
   Flask==2.0.3
   Flask-AppBuilder==4.1.6
   Flask-Babel==2.0.0
   Flask-Caching==1.11.1
   Flask-Compress==1.13
   Flask-JWT-Extended==4.4.4
   Flask-Login==0.6.2
   Flask-Migrate==4.0.0
   Flask-OpenID==1.3.0
   Flask-SQLAlchemy==2.5.1
   flask-talisman==1.0.0
   Flask-WTF==0.14.3
   flower==1.2.0
   frozenlist==1.3.3
   func-timeout==4.3.5
   future==0.18.2
   geographiclib==2.0
   geopy==2.3.0
   gevent==22.10.2
   graphlib-backport==1.0.3
   greenlet==2.0.1
   gunicorn==20.1.0
   h11==0.14.0
   hashids==1.3.1
   hijri-converter==2.2.4
   holidays==0.14.2
   humanize==4.4.0
   idna==3.4
   importlib-metadata==5.2.0
   importlib-resources==5.10.2
   infi.clickhouse-orm==2.1.3
   ipdb==0.13.11
   ipython==8.7.0
   iso8601==1.1.0
   isodate==0.6.1
   itsdangerous==2.1.2
   jedi==0.18.2
   Jinja2==3.0.0
   jmespath==1.0.1
   jsonschema==4.17.3
   kombu==5.2.4
   korean-lunar-calendar==0.3.1
   Mako==1.2.4
   Markdown==3.4.1
   MarkupSafe==2.1.1
   marshmallow==3.19.0
   marshmallow-enum==1.5.1
   marshmallow-sqlalchemy==0.26.1
   matplotlib-inline==0.1.6
   msgpack==1.0.4
   multidict==6.0.4
   mysqlclient==2.1.1
   numpy==1.22.1
   outcome==1.2.0
   packaging==22.0
   pandas==1.3.5
   parsedatetime==2.6
   parso==0.8.3
   pexpect==4.8.0
   pgsanity==0.2.9
   pickleshare==0.7.5
   pkgutil_resolve_name==1.3.10
   polyline==1.4.0
   prison==0.2.1
   prometheus-client==0.15.0
   prompt-toolkit==3.0.36
   psycopg2-binary==2.9.5
   ptyprocess==0.7.0
   pure-eval==0.2.2
   pure-sasl==0.6.2
   pyarrow==5.0.0
   pycparser==2.21
   Pygments==2.13.0
   PyJWT==2.6.0
   PyMeeus==0.5.12
   pyodbc==4.0.35
   pyparsing==3.0.9
   pyrsistent==0.19.2
   PySocks==1.7.1
   python-dateutil==2.8.2
   python-dotenv==0.21.0
   python-geohash==0.8.5
   python3-openid==3.2.0
   pytz==2022.7
   PyYAML==6.0
   redis==4.4.0
   requests==2.28.1
   s3transfer==0.6.0
   selenium==4.7.2
   simplejson==3.18.1
   six==1.16.0
   slackclient==2.5.0
   sniffio==1.3.0
   sortedcontainers==2.4.0
   SQLAlchemy==1.3.24
   SQLAlchemy-Utils==0.37.9
   sqlparse==0.3.0
   stack-data==0.6.2
   tabulate==0.8.9
   tomli==2.0.1
   tornado==6.2
   traitlets==5.8.0
   trino==0.320.0
   trio==0.22.0
   trio-websocket==0.9.2
   typing-extensions==3.10.0.2
   urllib3==1.26.13
   vertica-python==0.9.1
   vine==5.0.0
   wcwidth==0.2.5
   webencodings==0.5.1
   Werkzeug==2.0.0
   wsproto==1.2.0
   WTForms==2.3.3
   WTForms-JSON==0.3.5
   yarl==1.8.2
   zipp==3.11.0
   zope.event==4.6
   zope.interface==5.5.2
   PyHive==0.6.5
   thrift==0.16.0
   thrift-sasl==0.4.3
   clickhouse-connect==0.4.8
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] leagueofsoups commented on issue #23546: superset 2.0.1 dashboard and charts async update

Posted by "leagueofsoups (via GitHub)" <gi...@apache.org>.
leagueofsoups commented on issue #23546:
URL: https://github.com/apache/superset/issues/23546#issuecomment-1493324160

   My debug results:
   My chart generate POST with body
   ```
   result_type:	"full"
   result_format: "json"
   ```
   When I have valid cache, my web pod return result from cache https://github.com/apache/superset/blame/master/superset/charts/data/api.py#L322 and response contain "cache_key" https://github.com/apache/superset/blob/master/superset/charts/data/commands/get_data_command.py#L63
   
   When my cache invalid, query run async on celery worker. https://github.com/apache/superset/blame/master/superset/charts/data/api.py#L338 
   https://github.com/seonguk9303/superset/blob/main/superset/charts/data/commands/create_async_job_command.py#L35
   
   But when I try click force update inside chart page, my result return from redis, is this expected?
   <img width="296" alt="изображение" src="https://user-images.githubusercontent.com/41836635/229353929-80094f18-fe4e-416f-9a16-4a23f7d0bc64.png">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] superset 2.0.1 dashboard and charts async update [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #23546:
URL: https://github.com/apache/superset/issues/23546#issuecomment-1962098993

   This version of Superset is no longer supported. If anyone is still encountering this in current versions (3.x) please open a new Issue or a PR to address the problem... or we can simply re-open this one :) 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] superset 2.0.1 dashboard and charts async update [superset]

Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #23546: superset 2.0.1 dashboard and charts async update
URL: https://github.com/apache/superset/issues/23546


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org