You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/03/23 12:42:25 UTC
[GitHub] [superset] srinisubramanian edited a comment on issue #19297: Export / Import fails with validation error
srinisubramanian edited a comment on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076323033
There is definitely an Issue with the database import. Even with the changed code the database import fails if the database and/or datasets are deleted and then the import is attempted. Error is a little different though,
Here is what I found:
1. Patch the code as above, export and import it works.
2. Delete the dashboard and charts and import again. It works.
3. Delete the dashboard, charts and datasets and import. Fails. The log is a little different now and has some MySQL syntax errors?
**Using MySQL 8 and pymysql to connect**
```
Importing new tables digital_forms.USERS
2022-03-23 17:56:50,171:INFO:superset.models.helpers:Importing new tables digital_forms.USERS
Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
2022-03-23 17:56:50,198:INFO:superset.models.helpers:Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
Importing new table_columns LAST_ACCESS_DATE
2022-03-23 17:56:50,224:INFO:superset.models.helpers:Importing new table_columns LAST_ACCESS_DATE
Error running import command
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
self._import(db.session, self._configs, self.overwrite)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
dataset = import_dataset(session, config, overwrite=False)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
child_class.import_from_dict(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
obj = obj_query.one_or_none()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
ret = list(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
self.session._autoflush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
[SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
[parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
(Background on this error at: http://sqlalche.me/e/13/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
raise self.import_error() from ex
superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
2022-03-23 17:56:50,243:ERROR:superset.dashboards.commands.importers.dispatcher:Error running import command
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
self._import(db.session, self._configs, self.overwrite)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
dataset = import_dataset(session, config, overwrite=False)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
child_class.import_from_dict(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
obj = obj_query.one_or_none()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
ret = list(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
self.session._autoflush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
[SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
[parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
(Background on this error at: http://sqlalche.me/e/13/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
raise self.import_error() from ex
superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
Import dashboard failed for an unknown reason
2022-03-23 17:56:50,271:WARNING:superset.views.base:Import dashboard failed for an unknown reason
2022-03-23 17:56:50,272:INFO:werkzeug:127.0.0.1 - - [23/Mar/2022 17:56:50] "POST /api/v1/dashboard/import/ HTTP/1.1" 500 -
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org