You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/02/25 07:02:52 UTC
[GitHub] [superset] iercan opened a new issue #12766: Global Async Queries doesn't work
iercan opened a new issue #12766:
URL: https://github.com/apache/superset/issues/12766
Hi.
I tried to enable GAQ feature with apache/superset:latest docker image. After I enabled it I experienced some dashboard stopped working and all I see is white screen. When I checked network activity I saw too much async request and they weren't stoping and returing emtpy results.
### Expected results
Dashboard should load faster and async
### Actual results
Charts and dashboards aren't loading. Also see this exception in worker
```
worker_1 | [2021-01-26 10:16:36,308: ERROR/ForkPoolWorker-11] Task load_explore_json_into_cache[069996ac-d5d9-4c29-a6e4-c99e3faf6899] raised unexpected: NoSuchColumnError("Could not locate column in row for column 'table_columns.id'")
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1 | payload = viz_obj.get_payload()
worker_1 | File "/app/superset/viz.py", line 476, in get_payload
worker_1 | columns = set(self.datasource.column_names)
worker_1 | File "/app/superset/connectors/base/models.py", line 145, in column_names
worker_1 | return sorted([c.column_name for c in self.columns], key=lambda x: x or "")
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 287, in __get__
worker_1 | return self.impl.get(instance_state(instance), dict_)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 723, in get
worker_1 | value = self.callable_(state, passive)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state
worker_1 | session, state, primary_key_identity, passive
worker_1 | File "<string>", line 1, in <lambda>
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 902, in _emit_lazyload
worker_1 | .with_post_criteria(set_default_params)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 544, in all
worker_1 | return list(self)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
worker_1 | cursor.close()
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
worker_1 | with_traceback=exc_tb,
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1 | raise exception
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
worker_1 | rows = [proc(row) for row in fetch]
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
worker_1 | rows = [proc(row) for row in fetch]
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in _instance
worker_1 | tuple([row[column] for column in pk_cols]),
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in <listcomp>
worker_1 | tuple([row[column] for column in pk_cols]),
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/result.py", line 686, in _key_fallback
worker_1 | replace_context=err,
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1 | raise exception
worker_1 | sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'table_columns.id'"
```
#### Screenshots
#### How to reproduce the bug
1. Create superset and celery environment with apache/superset:latest image
2. Enable GAQ feature and configure secret.
### Environment
- superset version: latest
- python version: 3.7
- mysql 5.7.32
- redis 6.0.9
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [X] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [X] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768268664
Today I made some more test. I disabled thumbnails which was enabled on other test. This time dashoard charts stuck on loading states and async_event requests started returning 401. On worker logs I see queries are executing. Really couldn't figure out what's wrong here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan edited a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan edited a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768829623
> * Which visualizations are triggering the errors?
Dashboard I tested we have filter box, line chart, time series chart, big number and table. Also we have defined metrics on that dataset.
> * It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?
Yes It come from druid but I got this error from mysql too.
```
worker_1 | [2021-01-28 06:17:13,108: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[fd7b0575-725e-4b19-925b-4f6530cfc478] raised unexpected: SupersetVizException('[{\'message\': \'"Could not locate column in row for column \\\'sql_metrics.id\\\'"\', \'error_type\': <SupersetErrorType.VIZ_GET_DF_ERROR: \'VIZ_GET_DF_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': None}]')
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1 | raise SupersetVizException(errors=payload["errors"])
worker_1 | superset.exceptions.SupersetVizException: [{'message': '"Could not locate column in row for column \'sql_metrics.id\'"', 'error_type': <SupersetErrorType.VIZ_GET_DF_ERROR: 'VIZ_GET_DF_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': None}]
```
> * Are you able to run asynchronous queries in SQL Lab?
Sql lab works fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768288306
Yet another
```
worker_1 | [2021-01-27 13:32:29,087: ERROR/ForkPoolWorker-1] Task load_explore_json_into_cache[60028736-ba7e-49e2-8513-f4a66d63f6b1] raised unexpected: TypeError("'NoneType' object does not support item assignment")
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1 | payload = viz_obj.get_payload()
worker_1 | File "/app/superset/viz.py", line 470, in get_payload
worker_1 | payload["data"] = self.get_data(df)
worker_1 | File "/app/superset/viz.py", line 2048, in get_data
worker_1 | df[col] = []
worker_1 | TypeError: 'NoneType' object does not support item assignment
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768280265
Permission error gone after I run `superset init`. But problem I mentioned on my first post still persist.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-789380255
@robdiciuccio I did my test with gevent and concurrency was 500. I used `Refresh dashboard` options which reload dashboard without cache.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] junlincc commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
junlincc commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-767743278
@robdiciuccio 🙏
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-771258568
I tested locally running async queries against a MySQL database, and am still unable to replicate the issue. All of the stack traces you've posted appear to have something to do with a missing column in the dataset you're querying. I'm not sure how async queries could be the culprit here, as the query execution code is fundamentally the same, but please do let us know if you find any more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-786483362
@robdiciuccio I also make some performance tests. I loaded a dashboard that have more than 20 charts. Looks like with GAQ dashboards loads far slower. Also chart loads not finish until second explode request, I don't know if this is expected behavior
with GAQ dashboard loads in 13 sec
without GAQ dashboard loads in 5 sec
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768283973
Here is another exception I catch. I don't know what is different with async and sync queries but looks like the way celery worker send queries cause these exceptions
```
worker_1 | [2021-01-27 13:21:35,231: ERROR/ForkPoolWorker-7] Task load_explore_json_into_cache[2f6fdc73-84e8-4eac-9c3e-dbad93852f49] raised unexpected: SupersetVizException('[{\'message\': "Unknown exception (org.apache.calcite.tools.ValidationException): org.apache.calcite.runtime.CalciteContextException: From line 7, column 10 to line 7, column 20: Column \'algorithm\' not found in any table", \'error_type\': <SupersetErrorType.GENERIC_DB_ENGINE_ERROR: \'GENERIC_DB_ENGINE_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': {\'engine_name\': \'Apache Druid\', \'issue_codes\': [{\'code\': 1002, \'message\': \'Issue 1002 - The database returned an unexpected error.\'}]}}]')
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1 | raise SupersetVizException(errors=payload["errors"])
worker_1 | superset.exceptions.SupersetVizException: [{'message': "Unknown exception (org.apache.calcite.tools.ValidationException): org.apache.calcite.runtime.CalciteContextException: From line 7, column 10 to line 7, column 20: Column 'algorithm' not found in any table", 'error_type': <SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': {'engine_name': 'Apache Druid', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]}}]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan closed issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan closed issue #12766:
URL: https://github.com/apache/superset/issues/12766
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan edited a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan edited a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-786483362
@robdiciuccio I also made some performance tests. I loaded a dashboard that have more than 20 charts. Looks like with GAQ dashboards loads far slower. Also chart loads not finish until second explode request, I don't know if this is expected behavior
with GAQ dashboard loads in 13 sec
without GAQ dashboard loads in 5 sec
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-771391509
I tried too many config combinations but unfortunately couldn't resolve issue. For now, I decided to disable that feature and go on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-786306860
@iercan can you test the fix in #13350 to see if that resolves the issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan edited a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan edited a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-789380255
@robdiciuccio I did my test with gevent and concurrency was 500. I used `Refresh dashboard` options which reload whole dashboard without cache.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-780969413
Thanks for the updates @iercan. Interesting that the errors were resolved when increasing the polling delay. I had this running on my Macbook at 200ms delay with no errors. What kind of hardware or virtual machine are you running this on?
Also, [websocket support](https://github.com/apache/superset/pull/11498) should be shipping soon, which will negate the need for polling.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan edited a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan edited a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768829623
> * Which visualizations are triggering the errors?
Dashboard I tested we have filter box, line chart, time series chart, big number and table. Also we have defined metrics on that dataset.
> * It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?
Yes It come from druid but I got below error from mysql. Looks like it is not druid spesific
```
worker_1 | [2021-01-28 06:17:13,108: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[fd7b0575-725e-4b19-925b-4f6530cfc478] raised unexpected: SupersetVizException('[{\'message\': \'"Could not locate column in row for column \\\'sql_metrics.id\\\'"\', \'error_type\': <SupersetErrorType.VIZ_GET_DF_ERROR: \'VIZ_GET_DF_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': None}]')
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1 | raise SupersetVizException(errors=payload["errors"])
worker_1 | superset.exceptions.SupersetVizException: [{'message': '"Could not locate column in row for column \'sql_metrics.id\'"', 'error_type': <SupersetErrorType.VIZ_GET_DF_ERROR: 'VIZ_GET_DF_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': None}]
```
> * Are you able to run asynchronous queries in SQL Lab?
Sql lab works fine.
> The same code is used to query the analytics DB in sync and async mode, so I'm also very curious why there's a discrepancy between the two. I should note that celery does not support hot reloads, so workers need to be restarted on any config changes.
I'm restarting all containers by using `docker-compose restart` whenever I change a config
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-779637626
@robdiciuccio Here is my final report. Chart load problem gone after I set `GLOBAL_ASYNC_QUERIES_POLLING_DELAY` from 500 to 1500. I observed dashboard load times with GAQ enabled and disabled. It seems while GAQ disabled dashboards loads faster. I believe it is because I had to increase pooling delay and our dashboards usually have 5-10 charts. As a result we decided to go on with GAQ disabled for now. I'd glad test it again after some improvements. Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio edited a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio edited a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-789283433
@iercan thanks for the report on performance. See my tests [here](https://github.com/apache/superset/pull/11499#issuecomment-742018892). It looks like you're using `prefork` with Celery; what's your concurrency setting (how many processes)?
Also, are these requests for cached charts, or a mix?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768032725
Hi @iercan, it is expected that the event polling mechanism makes continuous requests to the `/api/v1/async_event/` endpoint (presumably what's in the above screenshot). The `sqlalchemy.exc.NoSuchColumnError` trace you posted above appears to be a SQL query error against your analytics DB. Does the same query function correctly in synchronous mode?
Also, not sure if you've seen this, but check out the documentation added here: https://github.com/apache/superset/pull/12573
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-790254016
@robdiciuccio I see. I prefer to use gevent because prefork leaves defunct webdriver processes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-778220821
@robdiciuccio I did some further tests. I tried to change celery execution pool from prefork to gevent and interestingly errors gone. But this time I started to experience more slow load times or even some charts can not load even if I don't see any error in logs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-790113711
@iercan in my tests, celery with `gevent` performed the worst, with `prefork` returning results comparable to synchronous operation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-786458442
@robdiciuccio I'm still getting errors.
```
worker_1 | [2021-02-26 07:05:27,835: INFO/ForkPoolWorker-7] Cache key: fd68af649f2858ffb1156862272ba704
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,836: INFO/ForkPoolWorker-7] Serving from cache
worker_1 | Cache key: 926fcd84c4ea3e42956ae51436269022
worker_1 | Cache key: a98643384e8bc016e02fe68b97211a32
worker_1 | [2021-02-26 07:05:27,840: INFO/ForkPoolWorker-1] Cache key: a98643384e8bc016e02fe68b97211a32
worker_1 | [2021-02-26 07:05:27,840: INFO/ForkPoolWorker-8] Cache key: 926fcd84c4ea3e42956ae51436269022
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,842: INFO/ForkPoolWorker-1] Serving from cache
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,842: INFO/ForkPoolWorker-8] Serving from cache
worker_1 | Cache key: e497d4025ce15e3ac7f4d1304bbfd1e8
worker_1 | [2021-02-26 07:05:27,846: INFO/ForkPoolWorker-10] Cache key: e497d4025ce15e3ac7f4d1304bbfd1e8
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,848: INFO/ForkPoolWorker-10] Serving from cache
worker_1 | [2021-02-26 07:05:27,854: INFO/ForkPoolWorker-1] Task load_explore_json_into_cache[530e7fcb-c6e6-448b-be9b-4ff0307dd745] succeeded in 0.019942453131079674s: None
worker_1 | Cache key: e3f5fbde933b2a77404ac5ba57910469
worker_1 | [2021-02-26 07:05:27,861: INFO/ForkPoolWorker-10] Cache key: e3f5fbde933b2a77404ac5ba57910469
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,862: INFO/ForkPoolWorker-10] Serving from cache
worker_1 | [2021-02-26 07:05:27,864: INFO/ForkPoolWorker-7] Task load_explore_json_into_cache[7e47b489-286a-43e4-9bb4-ff5706aa5050] succeeded in 0.0350632518529892s: None
worker_1 | [2021-02-26 07:05:27,871: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[53952638-59ac-481b-b30f-381469c1b4c8] raised unexpected: NoSuchColumnError("Could not locate column in row for column 'table_columns.id'")
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1 | payload = viz_obj.get_payload()
worker_1 | File "/app/superset/viz.py", line 454, in get_payload
worker_1 | columns = set(self.datasource.column_names)
worker_1 | File "/app/superset/connectors/base/models.py", line 145, in column_names
worker_1 | return sorted([c.column_name for c in self.columns], key=lambda x: x or "")
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 287, in __get__
worker_1 | return self.impl.get(instance_state(instance), dict_)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 723, in get
worker_1 | value = self.callable_(state, passive)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state
worker_1 | session, state, primary_key_identity, passive
worker_1 | File "<string>", line 1, in <lambda>
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 902, in _emit_lazyload
worker_1 | .with_post_criteria(set_default_params)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 544, in all
worker_1 | return list(self)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
worker_1 | cursor.close()
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
worker_1 | with_traceback=exc_tb,
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1 | raise exception
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
worker_1 | rows = [proc(row) for row in fetch]
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
worker_1 | rows = [proc(row) for row in fetch]
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in _instance
worker_1 | tuple([row[column] for column in pk_cols]),
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in <listcomp>
worker_1 | tuple([row[column] for column in pk_cols]),
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/result.py", line 686, in _key_fallback
worker_1 | replace_context=err,
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1 | raise exception
worker_1 | sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'table_columns.id'"
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-789283433
@iercan thanks for the report on performance. See my tests [here](https://github.com/apache/superset/pull/11499#issuecomment-742018892). It looks like you're using `prefork` with Celery; what's your concurrency setting (how many processes)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-771391509
I tried too many config combinations but unfortunately couldn't resolve issue. For now, I decided to disable that feature and go on.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan edited a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan edited a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-786483362
@robdiciuccio I also made some performance tests. I loaded a dashboard that have more than 20 charts. Looks like with GAQ dashboards loads far slower. Also chart loads does not finish until second explode request, I don't know if this is expected behavior
with GAQ dashboard loads in 13 sec
without GAQ dashboard loads in 5 sec
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768066711
@robdiciuccio There is no problem with synchronous mode. I already checked documentation and configured accordingly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-786471907
Sorry I realized I made test from wrong branch thats why I removed previous answer. I retested it with both prefork and gevent pools and it looks fine. Thanks for the fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-785654638
I've been able to reproduce the issue using Celery `prefork` concurrency. It seems to be the same issue documented here: https://github.com/apache/superset/issues/10530
Continuing to investigate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-781092531
We deployed superset via docker on a dedicated server which has amd cpu. As a client I'm using ubuntu 20.04 with chrome also have amd cpu.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan closed issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan closed issue #12766:
URL: https://github.com/apache/superset/issues/12766
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768499578
@iercan can you provide some additional information about your situation?
- Which visualizations are triggering the errors?
- It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?
- Are you able to run asynchronous queries in SQL Lab?
The same code is used to query the analytics DB in sync and async mode, so I'm also very curious why there's a discrepancy between the two. I should note that celery does not support hot reloads, so workers need to be restarted on any config changes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan removed a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan removed a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-786458442
@robdiciuccio I'm still getting errors.
```
worker_1 | [2021-02-26 07:05:27,835: INFO/ForkPoolWorker-7] Cache key: fd68af649f2858ffb1156862272ba704
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,836: INFO/ForkPoolWorker-7] Serving from cache
worker_1 | Cache key: 926fcd84c4ea3e42956ae51436269022
worker_1 | Cache key: a98643384e8bc016e02fe68b97211a32
worker_1 | [2021-02-26 07:05:27,840: INFO/ForkPoolWorker-1] Cache key: a98643384e8bc016e02fe68b97211a32
worker_1 | [2021-02-26 07:05:27,840: INFO/ForkPoolWorker-8] Cache key: 926fcd84c4ea3e42956ae51436269022
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,842: INFO/ForkPoolWorker-1] Serving from cache
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,842: INFO/ForkPoolWorker-8] Serving from cache
worker_1 | Cache key: e497d4025ce15e3ac7f4d1304bbfd1e8
worker_1 | [2021-02-26 07:05:27,846: INFO/ForkPoolWorker-10] Cache key: e497d4025ce15e3ac7f4d1304bbfd1e8
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,848: INFO/ForkPoolWorker-10] Serving from cache
worker_1 | [2021-02-26 07:05:27,854: INFO/ForkPoolWorker-1] Task load_explore_json_into_cache[530e7fcb-c6e6-448b-be9b-4ff0307dd745] succeeded in 0.019942453131079674s: None
worker_1 | Cache key: e3f5fbde933b2a77404ac5ba57910469
worker_1 | [2021-02-26 07:05:27,861: INFO/ForkPoolWorker-10] Cache key: e3f5fbde933b2a77404ac5ba57910469
worker_1 | Serving from cache
worker_1 | [2021-02-26 07:05:27,862: INFO/ForkPoolWorker-10] Serving from cache
worker_1 | [2021-02-26 07:05:27,864: INFO/ForkPoolWorker-7] Task load_explore_json_into_cache[7e47b489-286a-43e4-9bb4-ff5706aa5050] succeeded in 0.0350632518529892s: None
worker_1 | [2021-02-26 07:05:27,871: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[53952638-59ac-481b-b30f-381469c1b4c8] raised unexpected: NoSuchColumnError("Could not locate column in row for column 'table_columns.id'")
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 84, in load_explore_json_into_cache
worker_1 | payload = viz_obj.get_payload()
worker_1 | File "/app/superset/viz.py", line 454, in get_payload
worker_1 | columns = set(self.datasource.column_names)
worker_1 | File "/app/superset/connectors/base/models.py", line 145, in column_names
worker_1 | return sorted([c.column_name for c in self.columns], key=lambda x: x or "")
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 287, in __get__
worker_1 | return self.impl.get(instance_state(instance), dict_)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 723, in get
worker_1 | value = self.callable_(state, passive)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state
worker_1 | session, state, primary_key_identity, passive
worker_1 | File "<string>", line 1, in <lambda>
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 902, in _emit_lazyload
worker_1 | .with_post_criteria(set_default_params)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 544, in all
worker_1 | return list(self)
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
worker_1 | cursor.close()
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
worker_1 | with_traceback=exc_tb,
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1 | raise exception
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
worker_1 | rows = [proc(row) for row in fetch]
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
worker_1 | rows = [proc(row) for row in fetch]
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in _instance
worker_1 | tuple([row[column] for column in pk_cols]),
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 524, in <listcomp>
worker_1 | tuple([row[column] for column in pk_cols]),
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/result.py", line 686, in _key_fallback
worker_1 | replace_context=err,
worker_1 | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
worker_1 | raise exception
worker_1 | sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'table_columns.id'"
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan edited a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan edited a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-786471907
@robdiciuccio Sorry I realized I made test from wrong branch thats why I removed previous answer. I retested it with both prefork and gevent pools and it looks fine. Thanks for the fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-771258568
I tested locally running async queries against a MySQL database, and am still unable to replicate the issue. All of the stack traces you've posted appear to have something to do with a missing column in the dataset you're querying. I'm not sure how async queries could be the culprit here, as the query execution code is fundamentally the same, but please do let us know if you find any more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan edited a comment on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan edited a comment on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768829623
> * Which visualizations are triggering the errors?
Dashboard I tested we have filter box, line chart, time series chart, big number and table. Also we have defined metrics on that dataset.
> * It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?
Yes It come from druid but I got this error from mysql too.
```
worker_1 | [2021-01-28 06:17:13,108: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[fd7b0575-725e-4b19-925b-4f6530cfc478] raised unexpected: SupersetVizException('[{\'message\': \'"Could not locate column in row for column \\\'sql_metrics.id\\\'"\', \'error_type\': <SupersetErrorType.VIZ_GET_DF_ERROR: \'VIZ_GET_DF_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': None}]')
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1 | raise SupersetVizException(errors=payload["errors"])
worker_1 | superset.exceptions.SupersetVizException: [{'message': '"Could not locate column in row for column \'sql_metrics.id\'"', 'error_type': <SupersetErrorType.VIZ_GET_DF_ERROR: 'VIZ_GET_DF_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': None}]
```
> * Are you able to run asynchronous queries in SQL Lab?
Sql lab works fine.
> The same code is used to query the analytics DB in sync and async mode, so I'm also very curious why there's a discrepancy between the two. I should note that celery does not support hot reloads, so workers need to be restarted on any config changes.
I'm restarting all containers by using `docker-compose restart` whenever I change a config
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] iercan commented on issue #12766: Global Async Queries doesn't work
Posted by GitBox <gi...@apache.org>.
iercan commented on issue #12766:
URL: https://github.com/apache/superset/issues/12766#issuecomment-768829623
> * Which visualizations are triggering the errors?
Dashboard I tested we have filter box, line chart, time series chart, big number and table. Also we have defined metrics on that datasource.
> * It appears from the stacktraces that you're querying an Apache Druid DB, is that correct? Are you seeing errors with other DB types?
Yes It come from druid but I got this error from mysql too.
```
worker_1 | [2021-01-28 06:17:13,108: ERROR/ForkPoolWorker-8] Task load_explore_json_into_cache[fd7b0575-725e-4b19-925b-4f6530cfc478] raised unexpected: SupersetVizException('[{\'message\': \'"Could not locate column in row for column \\\'sql_metrics.id\\\'"\', \'error_type\': <SupersetErrorType.VIZ_GET_DF_ERROR: \'VIZ_GET_DF_ERROR\'>, \'level\': <ErrorLevel.ERROR: \'error\'>, \'extra\': None}]')
worker_1 | Traceback (most recent call last):
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 412, in trace_task
worker_1 | R = retval = fun(*args, **kwargs)
worker_1 | File "/app/superset/app.py", line 116, in __call__
worker_1 | return task_base.__call__(self, *args, **kwargs)
worker_1 | File "/usr/local/lib/python3.7/site-packages/celery/app/trace.py", line 704, in __protected_call__
worker_1 | return self.run(*args, **kwargs)
worker_1 | File "/app/superset/tasks/async_queries.py", line 108, in load_explore_json_into_cache
worker_1 | raise exc
worker_1 | File "/app/superset/tasks/async_queries.py", line 86, in load_explore_json_into_cache
worker_1 | raise SupersetVizException(errors=payload["errors"])
worker_1 | superset.exceptions.SupersetVizException: [{'message': '"Could not locate column in row for column \'sql_metrics.id\'"', 'error_type': <SupersetErrorType.VIZ_GET_DF_ERROR: 'VIZ_GET_DF_ERROR'>, 'level': <ErrorLevel.ERROR: 'error'>, 'extra': None}]
```
> * Are you able to run asynchronous queries in SQL Lab?
Sql lab works fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org