You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2019/03/29 09:02:07 UTC
[GitHub] [incubator-superset] nladuguie opened a new issue #7169:
IntegrityError when running long query on SQL Lab
nladuguie opened a new issue #7169: IntegrityError when running long query on SQL Lab
URL: https://github.com/apache/incubator-superset/issues/7169
Hi everybody,
We use Superset (incubator) since a few weeks now, and we use it to execute queries on PrestoDB cluster.
But we encounter a problem while running a basic `SELECT` query which is lasting about 4 to 5 minutes (so a long one).
The query is run asynchronously but the same error appears while running the same query synchronously.
### Superset version
0.28.1
### Expected results
The query runs until success and the result is displayed in the SQL Lab interface.
### Actual results
The HTTP request to the `/sql_json` API is failing with the `500` status code, around 3 to 4 minutes after the query is executed.
However, the query executed in PrestoDB cluster is completed after 4 to 5 minutes. That is to say the problem is not on the PrestoDB cluster side.
The stack trace resulting is the following :
```
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 226, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorvalue
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 217, in execute
res = self._query(query)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 378, in _query
rowcount = self._do_query(q)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 341, in _do_query
db.query(q)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 280, in query
_mysql.connection.query(self, query)
_mysql_exceptions.IntegrityError: (1062, "Duplicate entry '2AhlO0vMH' for key 'client_id'")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.6/site-packages/flask_appbuilder/security/decorators.py", line 52, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/superset/models/core.py", line 1017, in wrapper
value = f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/superset/views/core.py", line 2509, in sql_json
session.flush()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2243, in flush
self._flush(objects)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2369, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2333, in _flush
flush_context.execute()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute
rec.execute(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute
uow
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
mapper, table, insert)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 866, in _emit_insert_statements
execute(statement, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 226, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorvalue
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 217, in execute
res = self._query(query)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 378, in _query
rowcount = self._do_query(q)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 341, in _do_query
db.query(q)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 280, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.IntegrityError: (_mysql_exceptions.IntegrityError) (1062, "Duplicate entry '2AhlO0vMH' for key 'client_id'") [SQL: 'INSERT INTO query (client_id, database_id, tmp_table_name, user_id, status, tab_name, sql_editor_id, `schema`, `sql`, select_sql, executed_sql, `limit`, limit_used, select_as_cta, select_as_cta_used, progress, `rows`, error_message, results_key, start_time, start_running_time, end_time, end_result_backend_time, tracking_url, changed_on) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'] [parameters: ('2AhlO0vMH', 2, '', '2', 'running', 'Untitled Query', 'kvGcgs0TC', None, "SELECT XXX'", None, None, None, 0, 0, 0, 0, None, None, None, 1553848436421.259, None, None, None, None, datetime.datetime(2019, 3, 29, 8, 33, 56, 423599))] (Background on this error at: http://sqlalche.me/e/gkpj)
```
### Steps to reproduce
Run a long execution time query (around than 4 to 5 minutes), and wait until the HTTP request fails.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org