You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/03/22 05:07:15 UTC
[GitHub] [superset] srinisubramanian opened a new issue #19297: Export / Import fails with validation error
srinisubramanian opened a new issue #19297:
URL: https://github.com/apache/superset/issues/19297
Export a dashboard with VERSIONED_EXPORT=true. Try importing on same instance or another instance and it fails with a command validation error.
1. Based on responses to #19268 from @villebro I reset my dev superset instance. Then created a dashboard from scratch and saved and exported it. I then tried to reimport it and it fails with a command validation error.
2. I then reset the instance again and then tried to reimport the dashboard on a brand new setup. Fails with same error. The log is set to DEBUG but not very helpful
```
2022-03-22 10:25:21,789:INFO:werkzeug: * Running on http://127.0.0.1:8088/ (Press CTRL+C to quit)
2022-03-22 10:25:29,502:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:29] "GET /dashboard/list/?pageIndex=0&sortColumn=changed_on_delta_humanized&sortOrder=desc&viewMode=table HTTP/1.1" 200 -
2022-03-22 10:25:31,722:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:31] "GET /api/v1/dashboard/_info?q=(keys:!(permissions)) HTTP/1.1" 200 -
2022-03-22 10:25:31,727:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:31] "GET /api/v1/dashboard/?q=(order_column:changed_on_delta_humanized,order_direction:desc,page:0,page_size:25) HTTP/1.1" 200 -
Command failed validation
2022-03-22 10:25:42,497:INFO:superset.dashboards.commands.importers.dispatcher:Command failed validation
Error importing dashboard
2022-03-22 10:25:42,498:WARNING:superset.views.base:Error importing dashboard
2022-03-22 10:25:42,501:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:42] "POST /api/v1/dashboard/import/ HTTP/1.1" 422 -
Command failed validation
2022-03-22 10:25:48,012:INFO:superset.dashboards.commands.importers.dispatcher:Command failed validation
Error importing dashboard
2022-03-22 10:25:48,013:WARNING:superset.views.base:Error importing dashboard
2022-03-22 10:25:48,016:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:48] "POST /api/v1/dashboard/import/ HTTP/1.1" 422 -
```
Details of dashboard:
1. Linked to 4 datasets (3 tables and a view) on MySQL
2. Has 2 tabs
3. Has markdown, big number with timeline, area and barcharts v2 with a filterbox
#### How to reproduce the bug
1. Create a dashboard, save and export it (VERSIONED_EXPORT = true)
2. Try to import and fails with a validation error
3. The issue is that a simple dashboard with just one or two charts seems to work. A more involved dashboard such as the one I am building fails
### Expected results
Import should be successful
### Actual results
Fails import
#### Screenshots
### Environment
(please complete the following information):
- browser type and version: Chrome
- superset version: `superset version` 1.4.1
- python version: `python --version` 3.8.12
- node.js version: `node -v`
- any feature flags active: VERSIONED_EXPORT
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [ ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [ ] I have reproduced the issue with at least the latest released version of superset.
- [ ] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
If required @villebro please let me know and I can email you an export of the database (data and superset) and a copy of the dashboard zip file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian closed issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian closed issue #19297:
URL: https://github.com/apache/superset/issues/19297
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1075044629
@EBoisseauSierra @betodealmeida can you help by letting me know where exactly was that code added? File info is there but not function/line info. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1074955097
Some additional info. I added a few prints of stacktraces to dispatcher and commands/importers/v1/__init__.py. Here is some additional info to trace this ..
```
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/commands/importers/v1/__init__.py", line 119, in validate
schema.load(config)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 719, in load
return self._do_load(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 904, in _do_load
raise exc
marshmallow.exceptions.ValidationError: {'columns': {0: {'extra': ['Not a valid mapping type.']}, 1: {'extra': ['Not a valid mapping type.']}, 2: {'extra': ['Not a valid mapping type.']}, 3: {'extra': ['Not a valid mapping type.']}, 4: {'extra': ['Not a valid mapping type.']}, 5: {'extra': ['Not a valid mapping type.']}, 6: {'extra': ['Not a valid mapping type.']}, 7: {'extra': ['Not a valid mapping type.']}, 8: {'extra': ['Not a valid mapping type.']}, 9: {'extra': ['Not a valid mapping type.']}, 10: {'extra': ['Not a valid mapping type.']}, 11: {'extra': ['Not a valid mapping type.']}}}
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/commands/importers/v1/__init__.py", line 119, in validate
schema.load(config)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 719, in load
return self._do_load(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 904, in _do_load
raise exc
marshmallow.exceptions.ValidationError: {'columns': {0: {'extra': ['Not a valid mapping type.']}, 1: {'extra': ['Not a valid mapping type.']}, 2: {'extra': ['Not a valid mapping type.']}, 3: {'extra': ['Not a valid mapping type.']}, 4: {'extra': ['Not a valid mapping type.']}, 5: {'extra': ['Not a valid mapping type.']}, 6: {'extra': ['Not a valid mapping type.']}, 7: {'extra': ['Not a valid mapping type.']}, 8: {'extra': ['Not a valid mapping type.']}, 9: {'extra': ['Not a valid mapping type.']}, 10: {'extra': ['Not a valid mapping type.']}, 11: {'extra': ['Not a valid mapping type.']}, 12: {'extra': ['Not a valid mapping type.']}, 13: {'extra': ['Not a valid mapping type.']}, 14: {'extra': ['Not a valid mapping type.']}, 15: {'extra': ['Not a valid mapping type.']}, 16: {'extra': ['Not a valid mapping type.']}, 17: {'extra': ['Not a valid mapping type.']}, 18: {'extra': ['Not a valid mapping type.']}, 19: {'extra': ['Not a valid mapping type.']}, 20: {'extra': ['Not a valid mapp
ing type.']}, 21: {'extra': ['Not a valid mapping type.']}, 22: {'extra': ['Not a valid mapping type.']}, 23: {'extra': ['Not a valid mapping type.']}, 24: {'extra': ['Not a valid mapping type.']}, 25: {'extra': ['Not a valid mapping type.']}, 26: {'extra': ['Not a valid mapping type.']}, 27: {'extra': ['Not a valid mapping type.']}}}
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/commands/importers/v1/__init__.py", line 119, in validate
schema.load(config)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 719, in load
return self._do_load(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 904, in _do_load
raise exc
marshmallow.exceptions.ValidationError: {'columns': {0: {'extra': ['Not a valid mapping type.']}, 1: {'extra': ['Not a valid mapping type.']}, 2: {'extra': ['Not a valid mapping type.']}, 3: {'extra': ['Not a valid mapping type.']}, 4: {'extra': ['Not a valid mapping type.']}, 5: {'extra': ['Not a valid mapping type.']}, 6: {'extra': ['Not a valid mapping type.']}, 7: {'extra': ['Not a valid mapping type.']}, 8: {'extra': ['Not a valid mapping type.']}, 9: {'extra': ['Not a valid mapping type.']}, 10: {'extra': ['Not a valid mapping type.']}, 11: {'extra': ['Not a valid mapping type.']}, 12: {'extra': ['Not a valid mapping type.']}, 13: {'extra': ['Not a valid mapping type.']}, 14: {'extra': ['Not a valid mapping type.']}, 15: {'extra': ['Not a valid mapping type.']}, 16: {'extra': ['Not a valid mapping type.']}, 17: {'extra': ['Not a valid mapping type.']}, 18: {'extra': ['Not a valid mapping type.']}}}
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/commands/importers/v1/__init__.py", line 119, in validate
schema.load(config)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 719, in load
return self._do_load(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 904, in _do_load
raise exc
marshmallow.exceptions.ValidationError: {'columns': {0: {'extra': ['Not a valid mapping type.']}, 1: {'extra': ['Not a valid mapping type.']}, 2: {'extra': ['Not a valid mapping type.']}, 3: {'extra': ['Not a valid mapping type.']}, 4: {'extra': ['Not a valid mapping type.']}, 5: {'extra': ['Not a valid mapping type.']}, 6: {'extra': ['Not a valid mapping type.']}, 7: {'extra': ['Not a valid mapping type.']}, 8: {'extra': ['Not a valid mapping type.']}, 9: {'extra': ['Not a valid mapping type.']}, 10: {'extra': ['Not a valid mapping type.']}, 11: {'extra': ['Not a valid mapping type.']}}}
Error importing dashboard
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 59, in run
command.run()
``
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1075087594
@EBoisseauSierra I guess its somehow related since instead of import dashboard if I try to import dataset it fails with the same error.
However in my case extra is NULL in the database table: tables
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1079586531
@betodealmeida My apologies for flagging this wrongly.. I did patch the export but didnt patch the import. Once I patched the import also (datasets/commands/importers/utils.py) it imported correctly.
I am closing this issue now as it seems to be fixed in master. Will create a new one in case of any other issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] EBoisseauSierra commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
EBoisseauSierra commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1075017956
Related to #19103?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1079088924
@villebro @srinify Can someone take a look and confirm the issue and what needs to be done? A workaround for now will also be acceptable while the issue is triaged and fixed. Appreciate your assistance in this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076350868
cc @betodealmeida @eschutho to take a look at this. The export/import is such an important feature and needs some tlc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076323033
There is definitely an Issue with the database import. Even with the changed code the database import fails if the database and/or datasets are deleted and then the import is attempted. Error is a little different though,
Here is what I found:
1. Patch the code as above, export and import it works.
2. Delete the dashboard and charts and import again. It works.
3. Delete the dashboard, charts and datasets and import. Fails. The log is a little different now and has some MySQL syntax errors?
```
Importing new tables digital_forms.USERS
2022-03-23 17:56:50,171:INFO:superset.models.helpers:Importing new tables digital_forms.USERS
Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
2022-03-23 17:56:50,198:INFO:superset.models.helpers:Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
Importing new table_columns LAST_ACCESS_DATE
2022-03-23 17:56:50,224:INFO:superset.models.helpers:Importing new table_columns LAST_ACCESS_DATE
Error running import command
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
self._import(db.session, self._configs, self.overwrite)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
dataset = import_dataset(session, config, overwrite=False)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
child_class.import_from_dict(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
obj = obj_query.one_or_none()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
ret = list(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
self.session._autoflush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
[SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
[parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
(Background on this error at: http://sqlalche.me/e/13/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
raise self.import_error() from ex
superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
2022-03-23 17:56:50,243:ERROR:superset.dashboards.commands.importers.dispatcher:Error running import command
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
self._import(db.session, self._configs, self.overwrite)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
dataset = import_dataset(session, config, overwrite=False)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
child_class.import_from_dict(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
obj = obj_query.one_or_none()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
ret = list(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
self.session._autoflush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
[SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
[parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
(Background on this error at: http://sqlalche.me/e/13/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
raise self.import_error() from ex
superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
Import dashboard failed for an unknown reason
2022-03-23 17:56:50,271:WARNING:superset.views.base:Import dashboard failed for an unknown reason
2022-03-23 17:56:50,272:INFO:werkzeug:127.0.0.1 - - [23/Mar/2022 17:56:50] "POST /api/v1/dashboard/import/ HTTP/1.1" 500 -
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian edited a comment on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian edited a comment on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076323033
There is definitely an Issue with the database import. Even with the changed code the database import fails if the database and/or datasets are deleted and then the import is attempted. Error is a little different though,
Here is what I found:
1. Patch the code as above, export and import it works.
2. Delete the dashboard and charts and import again. It works.
3. Delete the dashboard, charts and datasets and import. Fails. The log is a little different now and has some MySQL syntax errors?
**Using MySQL 8 and pymysql to connect**
```
Importing new tables digital_forms.USERS
2022-03-23 17:56:50,171:INFO:superset.models.helpers:Importing new tables digital_forms.USERS
Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
2022-03-23 17:56:50,198:INFO:superset.models.helpers:Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
Importing new table_columns LAST_ACCESS_DATE
2022-03-23 17:56:50,224:INFO:superset.models.helpers:Importing new table_columns LAST_ACCESS_DATE
Error running import command
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
self._import(db.session, self._configs, self.overwrite)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
dataset = import_dataset(session, config, overwrite=False)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
child_class.import_from_dict(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
obj = obj_query.one_or_none()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
ret = list(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
self.session._autoflush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
[SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
[parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
(Background on this error at: http://sqlalche.me/e/13/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
raise self.import_error() from ex
superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
2022-03-23 17:56:50,243:ERROR:superset.dashboards.commands.importers.dispatcher:Error running import command
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
self._import(db.session, self._configs, self.overwrite)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
dataset = import_dataset(session, config, overwrite=False)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
child_class.import_from_dict(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
obj = obj_query.one_or_none()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
ret = list(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
self.session._autoflush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
[SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
[parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
(Background on this error at: http://sqlalche.me/e/13/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
raise self.import_error() from ex
superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
Import dashboard failed for an unknown reason
2022-03-23 17:56:50,271:WARNING:superset.views.base:Import dashboard failed for an unknown reason
2022-03-23 17:56:50,272:INFO:werkzeug:127.0.0.1 - - [23/Mar/2022 17:56:50] "POST /api/v1/dashboard/import/ HTTP/1.1" 500 -
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian edited a comment on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian edited a comment on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076323033
There is definitely an Issue with the database import. Even with the changed code the database import fails if the database and/or datasets are deleted and then the import is attempted. Error I think is still to do with **extra** but now probably due to the fact that extra is set as {} instead of '{}'. So looks like for validation while the fix seems to work by removing the '{}' and making it {}, for saving/inserting the '{}' are obviously needed.
Here is what I found:
1. Patch the code as above, export and import it works.
2. Delete the dashboard and charts and import again. It works.
3. Delete the dashboard, charts and datasets and import. Fails. The log is a little different now and has some MySQL syntax errors?
**Using MySQL 8 and pymysql to connect**
```
Importing new tables digital_forms.USERS
2022-03-23 17:56:50,171:INFO:superset.models.helpers:Importing new tables digital_forms.USERS
Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
2022-03-23 17:56:50,198:INFO:superset.models.helpers:Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
Importing new table_columns LAST_ACCESS_DATE
2022-03-23 17:56:50,224:INFO:superset.models.helpers:Importing new table_columns LAST_ACCESS_DATE
Error running import command
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
self._import(db.session, self._configs, self.overwrite)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
dataset = import_dataset(session, config, overwrite=False)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
child_class.import_from_dict(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
obj = obj_query.one_or_none()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
ret = list(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
self.session._autoflush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
[SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
[parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
(Background on this error at: http://sqlalche.me/e/13/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
raise self.import_error() from ex
superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
2022-03-23 17:56:50,243:ERROR:superset.dashboards.commands.importers.dispatcher:Error running import command
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
self._import(db.session, self._configs, self.overwrite)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
dataset = import_dataset(session, config, overwrite=False)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
child_class.import_from_dict(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
obj = obj_query.one_or_none()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
ret = list(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
self.session._autoflush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
[SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
[parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
(Background on this error at: http://sqlalche.me/e/13/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
raise self.import_error() from ex
superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
Import dashboard failed for an unknown reason
2022-03-23 17:56:50,271:WARNING:superset.views.base:Import dashboard failed for an unknown reason
2022-03-23 17:56:50,272:INFO:werkzeug:127.0.0.1 - - [23/Mar/2022 17:56:50] "POST /api/v1/dashboard/import/ HTTP/1.1" 500 -
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1075183348
OK .. more progress and looking good.
Based on comments in the other referenced issue (above), I replaced the code in datasets/commands/export.py from the master branch and it works!! So looks like solution is already in master and this does not work in 1.4.1 only
old code:
```
for metric in payload.get("metrics", []):
if metric.get("extra"):
try:
metric["extra"] = json.loads(metric["extra"])
except json.decoder.JSONDecodeError:
logger.info("Unable to decode `extra` field: %s", metric["extra"])
```
replaced with (from master):
```
for key in ("metrics", "columns"):
for attributes in payload.get(key, []):
if attributes.get("extra"):
try:
attributes["extra"] = json.loads(attributes["extra"])
except json.decoder.JSONDecodeError:
logger.info(
"Unable to decode `extra` field: %s", attributes["extra"]
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on issue #19297: Export / Import fails with validation error
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1079140117
Taking a look, sorry for the delay.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org