You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "leagueofsoups (via GitHub)" <gi...@apache.org> on 2023/03/31 17:07:07 UTC
[GitHub] [superset] leagueofsoups opened a new issue, #23546: superset 2.0.1 dashboard and charts async update
leagueofsoups opened a new issue, #23546:
URL: https://github.com/apache/superset/issues/23546
I can't celery workers for update dashboard and charts
#### How to reproduce the bug
1. Use this configuration fragment
```
class CeleryConfig(object):
broker_url = "redis://<fix_it>/0"
imports = (
'superset.sql_lab',
'superset.tasks',
'superset.tasks.thumbnails',
'superset.charts',
'superset.dashboards'
)
result_backend = "redis://<fix_it>/0"
worker_log_level = 'DEBUG'
worker_prefetch_multiplier = 10
task_acks_late = True
task_annotations = {
"tasks.add": {"rate_limit": "10/s"},
"sql_lab.get_sql_results": {"rate_limit": "100/s"},
"email_reports.send": {
"rate_limit": "1/s",
"time_limit": 120,
"soft_time_limit": 150,
"ignore_result": True,
},
}
beat_schedule = {
"email_reports.schedule_hourly": {
"task": "email_reports.schedule_hourly",
"schedule": crontab(minute=1, hour="*"),
},
}
CELERY_CONFIG = CeleryConfig
```
2. Add async parametres
```
GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
"port": 6379,
"host": "<FIXIT>",
"password": "<FIXIT>",
"db": 3,
"ssl": False,
}
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX = "async-events-"
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT = 1000
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT_FIREHOSE = 1000000
GLOBAL_ASYNC_QUERIES_JWT_COOKIE_NAME = "async-token"
GLOBAL_ASYNC_QUERIES_JWT_COOKIE_SECURE = False
GLOBAL_ASYNC_QUERIES_JWT_COOKIE_DOMAIN = None
GLOBAL_ASYNC_QUERIES_JWT_SECRET = "<FIXIT>"
GLOBAL_ASYNC_QUERIES_TRANSPORT = "polling"
GLOBAL_ASYNC_QUERIES_POLLING_DELAY = int(
timedelta(milliseconds=500).total_seconds() * 1000
)
```
3. add flag
```
FEATURE_FLAGS = {
'GLOBAL_ASYNC_QUERIES': True,
...
```
4. enable Advanced > Performance > Asynchronous query execution for datasource
5. open dashboards/chard and use this datasource (force refrash)
### Expected results
Found async tasks in celery
### Actual results
Have only this tasks names:
load_chart_data_into_cache
sql_lab.get_sql_results
load_explore_json_into_cache
### Environment
- browser type and version: any
- superset version: `2.0.1`
- python version: `Python 3.8.13`
- node.js version: `without node js`
- any feature flags active: GLOBAL_ASYNC_QUERIES + ENABLE_TEMPLATE_PROCESSING
### Checklist
- [ ] I Have async tasks
### Additional context
pip freeze
```
aiohttp==3.8.3
aiosignal==1.3.1
alembic==1.9.1
amqp==5.1.1
apache-superset==2.0.1
apispec==3.3.2
asttokens==2.2.1
async-generator==1.10
async-timeout==4.0.2
attrs==22.2.0
Authlib==1.2.0
Babel==2.11.0
backcall==0.2.0
backoff==2.2.1
billiard==3.6.4.0
bitarray==2.6.1
bleach==3.3.1
boto3==1.26.39
botocore==1.29.39
Brotli==1.0.9
cachelib==0.4.1
celery==5.2.2
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==2.0.12
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorama==0.4.6
convertdate==2.4.0
cron-descriptor==1.2.32
croniter==1.3.8
cryptography==38.0.2
decorator==5.1.1
defusedxml==0.7.1
deprecation==2.1.0
dnspython==2.2.1
email-validator==1.3.0
exceptiongroup==1.1.0
executing==1.2.0
Flask==2.0.3
Flask-AppBuilder==4.1.6
Flask-Babel==2.0.0
Flask-Caching==1.11.1
Flask-Compress==1.13
Flask-JWT-Extended==4.4.4
Flask-Login==0.6.2
Flask-Migrate==4.0.0
Flask-OpenID==1.3.0
Flask-SQLAlchemy==2.5.1
flask-talisman==1.0.0
Flask-WTF==0.14.3
flower==1.2.0
frozenlist==1.3.3
func-timeout==4.3.5
future==0.18.2
geographiclib==2.0
geopy==2.3.0
gevent==22.10.2
graphlib-backport==1.0.3
greenlet==2.0.1
gunicorn==20.1.0
h11==0.14.0
hashids==1.3.1
hijri-converter==2.2.4
holidays==0.14.2
humanize==4.4.0
idna==3.4
importlib-metadata==5.2.0
importlib-resources==5.10.2
infi.clickhouse-orm==2.1.3
ipdb==0.13.11
ipython==8.7.0
iso8601==1.1.0
isodate==0.6.1
itsdangerous==2.1.2
jedi==0.18.2
Jinja2==3.0.0
jmespath==1.0.1
jsonschema==4.17.3
kombu==5.2.4
korean-lunar-calendar==0.3.1
Mako==1.2.4
Markdown==3.4.1
MarkupSafe==2.1.1
marshmallow==3.19.0
marshmallow-enum==1.5.1
marshmallow-sqlalchemy==0.26.1
matplotlib-inline==0.1.6
msgpack==1.0.4
multidict==6.0.4
mysqlclient==2.1.1
numpy==1.22.1
outcome==1.2.0
packaging==22.0
pandas==1.3.5
parsedatetime==2.6
parso==0.8.3
pexpect==4.8.0
pgsanity==0.2.9
pickleshare==0.7.5
pkgutil_resolve_name==1.3.10
polyline==1.4.0
prison==0.2.1
prometheus-client==0.15.0
prompt-toolkit==3.0.36
psycopg2-binary==2.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pure-sasl==0.6.2
pyarrow==5.0.0
pycparser==2.21
Pygments==2.13.0
PyJWT==2.6.0
PyMeeus==0.5.12
pyodbc==4.0.35
pyparsing==3.0.9
pyrsistent==0.19.2
PySocks==1.7.1
python-dateutil==2.8.2
python-dotenv==0.21.0
python-geohash==0.8.5
python3-openid==3.2.0
pytz==2022.7
PyYAML==6.0
redis==4.4.0
requests==2.28.1
s3transfer==0.6.0
selenium==4.7.2
simplejson==3.18.1
six==1.16.0
slackclient==2.5.0
sniffio==1.3.0
sortedcontainers==2.4.0
SQLAlchemy==1.3.24
SQLAlchemy-Utils==0.37.9
sqlparse==0.3.0
stack-data==0.6.2
tabulate==0.8.9
tomli==2.0.1
tornado==6.2
traitlets==5.8.0
trino==0.320.0
trio==0.22.0
trio-websocket==0.9.2
typing-extensions==3.10.0.2
urllib3==1.26.13
vertica-python==0.9.1
vine==5.0.0
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.0.0
wsproto==1.2.0
WTForms==2.3.3
WTForms-JSON==0.3.5
yarl==1.8.2
zipp==3.11.0
zope.event==4.6
zope.interface==5.5.2
PyHive==0.6.5
thrift==0.16.0
thrift-sasl==0.4.3
clickhouse-connect==0.4.8
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] leagueofsoups commented on issue #23546: superset 2.0.1 dashboard and charts async update
Posted by "leagueofsoups (via GitHub)" <gi...@apache.org>.
leagueofsoups commented on issue #23546:
URL: https://github.com/apache/superset/issues/23546#issuecomment-1493324160
My debug results:
My chart generate POST with body
```
result_type: "full"
result_format: "json"
```
When I have valid cache, my web pod return result from cache https://github.com/apache/superset/blame/master/superset/charts/data/api.py#L322 and response contain "cache_key" https://github.com/apache/superset/blob/master/superset/charts/data/commands/get_data_command.py#L63
When my cache invalid, query run async on celery worker. https://github.com/apache/superset/blame/master/superset/charts/data/api.py#L338
https://github.com/seonguk9303/superset/blob/main/superset/charts/data/commands/create_async_job_command.py#L35
But when I try click force update inside chart page, my result return from redis, is this expected?
<img width="296" alt="изображение" src="https://user-images.githubusercontent.com/41836635/229353929-80094f18-fe4e-416f-9a16-4a23f7d0bc64.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] superset 2.0.1 dashboard and charts async update [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #23546:
URL: https://github.com/apache/superset/issues/23546#issuecomment-1962098993
This version of Superset is no longer supported. If anyone is still encountering this in current versions (3.x) please open a new Issue or a PR to address the problem... or we can simply re-open this one :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] superset 2.0.1 dashboard and charts async update [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #23546: superset 2.0.1 dashboard and charts async update
URL: https://github.com/apache/superset/issues/23546
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org