You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/03/22 05:07:15 UTC

[GitHub] [superset] srinisubramanian opened a new issue #19297: Export / Import fails with validation error

srinisubramanian opened a new issue #19297:
URL: https://github.com/apache/superset/issues/19297


   Export a dashboard with VERSIONED_EXPORT=true.  Try importing on same instance or another instance and it fails with a command validation error.
   
   1.  Based on responses to #19268 from @villebro I reset my dev superset instance.  Then created a dashboard from scratch and saved and exported it.  I then tried to reimport it and it fails with a command validation error.
   2. I then reset the instance again and then tried to reimport the dashboard on a brand new setup. Fails with same error.  The log is set to DEBUG but not very helpful
   
   ```
   2022-03-22 10:25:21,789:INFO:werkzeug: * Running on http://127.0.0.1:8088/ (Press CTRL+C to quit)
   2022-03-22 10:25:29,502:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:29] "GET /dashboard/list/?pageIndex=0&sortColumn=changed_on_delta_humanized&sortOrder=desc&viewMode=table HTTP/1.1" 200 -
   2022-03-22 10:25:31,722:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:31] "GET /api/v1/dashboard/_info?q=(keys:!(permissions)) HTTP/1.1" 200 -
   2022-03-22 10:25:31,727:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:31] "GET /api/v1/dashboard/?q=(order_column:changed_on_delta_humanized,order_direction:desc,page:0,page_size:25) HTTP/1.1" 200 -
   Command failed validation
   2022-03-22 10:25:42,497:INFO:superset.dashboards.commands.importers.dispatcher:Command failed validation
   Error importing dashboard
   2022-03-22 10:25:42,498:WARNING:superset.views.base:Error importing dashboard
   2022-03-22 10:25:42,501:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:42] "POST /api/v1/dashboard/import/ HTTP/1.1" 422 -
   Command failed validation
   2022-03-22 10:25:48,012:INFO:superset.dashboards.commands.importers.dispatcher:Command failed validation
   Error importing dashboard
   2022-03-22 10:25:48,013:WARNING:superset.views.base:Error importing dashboard
   2022-03-22 10:25:48,016:INFO:werkzeug:127.0.0.1 - - [22/Mar/2022 10:25:48] "POST /api/v1/dashboard/import/ HTTP/1.1" 422 -
   ```
   Details of dashboard:
   1.  Linked to 4 datasets (3 tables and a view) on MySQL
   2. Has 2 tabs
   3. Has markdown, big number with timeline, area and barcharts v2 with a filterbox
   
   #### How to reproduce the bug
   
   1. Create a dashboard, save and export it (VERSIONED_EXPORT = true)
   2. Try to import and fails with a validation error
   3. The issue is that a simple dashboard with just one or two charts seems to work.  A more involved dashboard such as the one I am building fails
   
   ### Expected results
   
   Import should be successful
   
   ### Actual results
   
   Fails import
   
   #### Screenshots
   
   
   ### Environment
   
   (please complete the following information):
   
   - browser type and version: Chrome
   - superset version: `superset version` 1.4.1
   - python version: `python --version` 3.8.12
   - node.js version: `node -v`
   - any feature flags active: VERSIONED_EXPORT
   
   ### Checklist
   
   Make sure to follow these steps before submitting your issue - thank you!
   
   - [ ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
   - [ ] I have reproduced the issue with at least the latest released version of superset.
   - [ ] I have checked the issue tracker for the same issue and I haven't found one similar.
   
   ### Additional context
   
   If required @villebro  please let me know and I can email you an export of the database (data and superset) and a copy of the dashboard zip file
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian closed issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian closed issue #19297:
URL: https://github.com/apache/superset/issues/19297


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1075044629


   @EBoisseauSierra @betodealmeida  can you help by letting me know where exactly was that code added?  File info is there but not function/line info.  Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1074955097


   Some additional info.  I added a few prints of stacktraces to dispatcher and commands/importers/v1/__init__.py.   Here is some additional info to trace this ..
   
   ```
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/commands/importers/v1/__init__.py", line 119, in validate
       schema.load(config)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 719, in load
       return self._do_load(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 904, in _do_load
       raise exc
   marshmallow.exceptions.ValidationError: {'columns': {0: {'extra': ['Not a valid mapping type.']}, 1: {'extra': ['Not a valid mapping type.']}, 2: {'extra': ['Not a valid mapping type.']}, 3: {'extra': ['Not a valid mapping type.']}, 4: {'extra': ['Not a valid mapping type.']}, 5: {'extra': ['Not a valid mapping type.']}, 6: {'extra': ['Not a valid mapping type.']}, 7: {'extra': ['Not a valid mapping type.']}, 8: {'extra': ['Not a valid mapping type.']}, 9: {'extra': ['Not a valid mapping type.']}, 10: {'extra': ['Not a valid mapping type.']}, 11: {'extra': ['Not a valid mapping type.']}}}
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/commands/importers/v1/__init__.py", line 119, in validate
       schema.load(config)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 719, in load
       return self._do_load(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 904, in _do_load
       raise exc
   marshmallow.exceptions.ValidationError: {'columns': {0: {'extra': ['Not a valid mapping type.']}, 1: {'extra': ['Not a valid mapping type.']}, 2: {'extra': ['Not a valid mapping type.']}, 3: {'extra': ['Not a valid mapping type.']}, 4: {'extra': ['Not a valid mapping type.']}, 5: {'extra': ['Not a valid mapping type.']}, 6: {'extra': ['Not a valid mapping type.']}, 7: {'extra': ['Not a valid mapping type.']}, 8: {'extra': ['Not a valid mapping type.']}, 9: {'extra': ['Not a valid mapping type.']}, 10: {'extra': ['Not a valid mapping type.']}, 11: {'extra': ['Not a valid mapping type.']}, 12: {'extra': ['Not a valid mapping type.']}, 13: {'extra': ['Not a valid mapping type.']}, 14: {'extra': ['Not a valid mapping type.']}, 15: {'extra': ['Not a valid mapping type.']}, 16: {'extra': ['Not a valid mapping type.']}, 17: {'extra': ['Not a valid mapping type.']}, 18: {'extra': ['Not a valid mapping type.']}, 19: {'extra': ['Not a valid mapping type.']}, 20: {'extra': ['Not a valid mapp
 ing type.']}, 21: {'extra': ['Not a valid mapping type.']}, 22: {'extra': ['Not a valid mapping type.']}, 23: {'extra': ['Not a valid mapping type.']}, 24: {'extra': ['Not a valid mapping type.']}, 25: {'extra': ['Not a valid mapping type.']}, 26: {'extra': ['Not a valid mapping type.']}, 27: {'extra': ['Not a valid mapping type.']}}}
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/commands/importers/v1/__init__.py", line 119, in validate
       schema.load(config)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 719, in load
       return self._do_load(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 904, in _do_load
       raise exc
   marshmallow.exceptions.ValidationError: {'columns': {0: {'extra': ['Not a valid mapping type.']}, 1: {'extra': ['Not a valid mapping type.']}, 2: {'extra': ['Not a valid mapping type.']}, 3: {'extra': ['Not a valid mapping type.']}, 4: {'extra': ['Not a valid mapping type.']}, 5: {'extra': ['Not a valid mapping type.']}, 6: {'extra': ['Not a valid mapping type.']}, 7: {'extra': ['Not a valid mapping type.']}, 8: {'extra': ['Not a valid mapping type.']}, 9: {'extra': ['Not a valid mapping type.']}, 10: {'extra': ['Not a valid mapping type.']}, 11: {'extra': ['Not a valid mapping type.']}, 12: {'extra': ['Not a valid mapping type.']}, 13: {'extra': ['Not a valid mapping type.']}, 14: {'extra': ['Not a valid mapping type.']}, 15: {'extra': ['Not a valid mapping type.']}, 16: {'extra': ['Not a valid mapping type.']}, 17: {'extra': ['Not a valid mapping type.']}, 18: {'extra': ['Not a valid mapping type.']}}}
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/commands/importers/v1/__init__.py", line 119, in validate
       schema.load(config)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 719, in load
       return self._do_load(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/marshmallow/schema.py", line 904, in _do_load
       raise exc
   marshmallow.exceptions.ValidationError: {'columns': {0: {'extra': ['Not a valid mapping type.']}, 1: {'extra': ['Not a valid mapping type.']}, 2: {'extra': ['Not a valid mapping type.']}, 3: {'extra': ['Not a valid mapping type.']}, 4: {'extra': ['Not a valid mapping type.']}, 5: {'extra': ['Not a valid mapping type.']}, 6: {'extra': ['Not a valid mapping type.']}, 7: {'extra': ['Not a valid mapping type.']}, 8: {'extra': ['Not a valid mapping type.']}, 9: {'extra': ['Not a valid mapping type.']}, 10: {'extra': ['Not a valid mapping type.']}, 11: {'extra': ['Not a valid mapping type.']}}}
   
   Error importing dashboard
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.9/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 59, in run
       command.run()
   ``


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1075087594


   @EBoisseauSierra I guess its somehow related since instead of import dashboard if I try to import dataset it fails with the same error.
   
   However in my case extra is NULL in the database table: tables


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1079586531


   @betodealmeida My apologies for flagging this wrongly.. I did patch the export but didnt patch the import.  Once I patched the import also (datasets/commands/importers/utils.py) it imported correctly.
   
   I am closing this issue now as it seems to be fixed in master.  Will create a new one in case of any other issues. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] EBoisseauSierra commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
EBoisseauSierra commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1075017956


   Related to  #19103?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1079088924


   @villebro @srinify Can someone take a look and confirm the issue and what needs to be done?  A workaround for now will also be acceptable while the issue is triaged and fixed.  Appreciate your assistance in this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076350868


   cc @betodealmeida @eschutho to take a look at this.  The export/import is such an important feature and needs some tlc


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076323033


   There is definitely an Issue with the database import.  Even with the changed code the database import fails if the database and/or datasets are deleted and then the import is attempted. Error is a little different though,
   
   Here is what I found:
   
   1.  Patch the code as above, export and import it works.
   2. Delete the dashboard and charts and  import again. It works.
   3. Delete the dashboard, charts and datasets and import.  Fails.  The log is a little different now and has some MySQL syntax errors?
   ```
   Importing new tables digital_forms.USERS
   2022-03-23 17:56:50,171:INFO:superset.models.helpers:Importing new tables digital_forms.USERS
   Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
   2022-03-23 17:56:50,198:INFO:superset.models.helpers:Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
   Importing new table_columns LAST_ACCESS_DATE
   2022-03-23 17:56:50,224:INFO:superset.models.helpers:Importing new table_columns LAST_ACCESS_DATE
   Error running import command
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
       self._import(db.session, self._configs, self.overwrite)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
       dataset = import_dataset(session, config, overwrite=False)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
       dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
       child_class.import_from_dict(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
       obj = obj_query.one_or_none()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
       ret = list(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
       self.session._autoflush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
       util.raise_(e, with_traceback=sys.exc_info()[2])
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
       self.flush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
       self._flush(objects)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
       transaction.rollback(_capture_exception=True)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
       compat.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
       flush_context.execute()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
       rec.execute(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
       persistence.save_obj(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
       _emit_insert_statements(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
       result = cached_connections[connection].execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
       return meth(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
       return connection._execute_clauseelement(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
       ret = self._execute_context(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
       self._handle_dbapi_exception(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
       util.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
   (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   [SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
   [parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
   (Background on this error at: http://sqlalche.me/e/13/f405)
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
       command.run()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
       raise self.import_error() from ex
   superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,243:ERROR:superset.dashboards.commands.importers.dispatcher:Error running import command
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
       self._import(db.session, self._configs, self.overwrite)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
       dataset = import_dataset(session, config, overwrite=False)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
       dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
       child_class.import_from_dict(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
       obj = obj_query.one_or_none()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
       ret = list(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
       self.session._autoflush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
       util.raise_(e, with_traceback=sys.exc_info()[2])
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
       self.flush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
       self._flush(objects)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
       transaction.rollback(_capture_exception=True)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
       compat.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
       flush_context.execute()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
       rec.execute(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
       persistence.save_obj(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
       _emit_insert_statements(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
       result = cached_connections[connection].execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
       return meth(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
       return connection._execute_clauseelement(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
       ret = self._execute_context(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
       self._handle_dbapi_exception(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
       util.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
   (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   [SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
   [parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
   (Background on this error at: http://sqlalche.me/e/13/f405)
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
       command.run()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
       raise self.import_error() from ex
   superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
   Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,271:WARNING:superset.views.base:Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,272:INFO:werkzeug:127.0.0.1 - - [23/Mar/2022 17:56:50] "POST /api/v1/dashboard/import/ HTTP/1.1" 500 -
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian edited a comment on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian edited a comment on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076323033


   There is definitely an Issue with the database import.  Even with the changed code the database import fails if the database and/or datasets are deleted and then the import is attempted. Error is a little different though,
   
   Here is what I found:
   
   1.  Patch the code as above, export and import it works.
   2. Delete the dashboard and charts and  import again. It works.
   3. Delete the dashboard, charts and datasets and import.  Fails.  The log is a little different now and has some MySQL syntax errors?
   
   **Using MySQL 8 and pymysql to connect**
   
   ```
   Importing new tables digital_forms.USERS
   2022-03-23 17:56:50,171:INFO:superset.models.helpers:Importing new tables digital_forms.USERS
   Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
   2022-03-23 17:56:50,198:INFO:superset.models.helpers:Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
   Importing new table_columns LAST_ACCESS_DATE
   2022-03-23 17:56:50,224:INFO:superset.models.helpers:Importing new table_columns LAST_ACCESS_DATE
   Error running import command
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
       self._import(db.session, self._configs, self.overwrite)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
       dataset = import_dataset(session, config, overwrite=False)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
       dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
       child_class.import_from_dict(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
       obj = obj_query.one_or_none()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
       ret = list(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
       self.session._autoflush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
       util.raise_(e, with_traceback=sys.exc_info()[2])
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
       self.flush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
       self._flush(objects)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
       transaction.rollback(_capture_exception=True)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
       compat.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
       flush_context.execute()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
       rec.execute(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
       persistence.save_obj(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
       _emit_insert_statements(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
       result = cached_connections[connection].execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
       return meth(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
       return connection._execute_clauseelement(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
       ret = self._execute_context(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
       self._handle_dbapi_exception(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
       util.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
   (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   [SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
   [parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
   (Background on this error at: http://sqlalche.me/e/13/f405)
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
       command.run()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
       raise self.import_error() from ex
   superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,243:ERROR:superset.dashboards.commands.importers.dispatcher:Error running import command
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
       self._import(db.session, self._configs, self.overwrite)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
       dataset = import_dataset(session, config, overwrite=False)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
       dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
       child_class.import_from_dict(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
       obj = obj_query.one_or_none()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
       ret = list(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
       self.session._autoflush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
       util.raise_(e, with_traceback=sys.exc_info()[2])
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
       self.flush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
       self._flush(objects)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
       transaction.rollback(_capture_exception=True)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
       compat.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
       flush_context.execute()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
       rec.execute(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
       persistence.save_obj(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
       _emit_insert_statements(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
       result = cached_connections[connection].execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
       return meth(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
       return connection._execute_clauseelement(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
       ret = self._execute_context(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
       self._handle_dbapi_exception(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
       util.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
   (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   [SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
   [parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
   (Background on this error at: http://sqlalche.me/e/13/f405)
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
       command.run()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
       raise self.import_error() from ex
   superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
   Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,271:WARNING:superset.views.base:Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,272:INFO:werkzeug:127.0.0.1 - - [23/Mar/2022 17:56:50] "POST /api/v1/dashboard/import/ HTTP/1.1" 500 -
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian edited a comment on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian edited a comment on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1076323033


   There is definitely an Issue with the database import.  Even with the changed code the database import fails if the database and/or datasets are deleted and then the import is attempted. Error I think is still to do with **extra** but now probably due to the fact that extra is set as {} instead of '{}'.  So looks like for validation while the fix seems to work by removing the '{}' and making it {}, for saving/inserting the '{}' are obviously needed.
   
   Here is what I found:
   
   1.  Patch the code as above, export and import it works.
   2. Delete the dashboard and charts and  import again. It works.
   3. Delete the dashboard, charts and datasets and import.  Fails.  The log is a little different now and has some MySQL syntax errors?
   
   **Using MySQL 8 and pymysql to connect**
   
   ```
   Importing new tables digital_forms.USERS
   2022-03-23 17:56:50,171:INFO:superset.models.helpers:Importing new tables digital_forms.USERS
   Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
   2022-03-23 17:56:50,198:INFO:superset.models.helpers:Importing new sql_metrics <superset.connectors.sqla.models.SqlMetric object at 0x123d108b0>
   Importing new table_columns LAST_ACCESS_DATE
   2022-03-23 17:56:50,224:INFO:superset.models.helpers:Importing new table_columns LAST_ACCESS_DATE
   Error running import command
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
       self._import(db.session, self._configs, self.overwrite)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
       dataset = import_dataset(session, config, overwrite=False)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
       dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
       child_class.import_from_dict(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
       obj = obj_query.one_or_none()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
       ret = list(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
       self.session._autoflush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
       util.raise_(e, with_traceback=sys.exc_info()[2])
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
       self.flush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
       self._flush(objects)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
       transaction.rollback(_capture_exception=True)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
       compat.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
       flush_context.execute()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
       rec.execute(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
       persistence.save_obj(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
       _emit_insert_statements(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
       result = cached_connections[connection].execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
       return meth(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
       return connection._execute_clauseelement(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
       ret = self._execute_context(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
       self._handle_dbapi_exception(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
       util.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
   (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   [SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
   [parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
   (Background on this error at: http://sqlalche.me/e/13/f405)
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
       command.run()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
       raise self.import_error() from ex
   superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,243:ERROR:superset.dashboards.commands.importers.dispatcher:Error running import command
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 69, in run
       self._import(db.session, self._configs, self.overwrite)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/v1/__init__.py", line 100, in _import
       dataset = import_dataset(session, config, overwrite=False)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/datasets/commands/importers/v1/utils.py", line 115, in import_dataset
       dataset = SqlaTable.import_from_dict(session, config, recursive=True, sync=sync)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 242, in import_from_dict
       child_class.import_from_dict(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/models/helpers.py", line 209, in import_from_dict
       obj = obj_query.one_or_none()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
       ret = list(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3534, in __iter__
       self.session._autoflush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
       util.raise_(e, with_traceback=sys.exc_info()[2])
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
       self.flush()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
       self._flush(objects)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
       transaction.rollback(_capture_exception=True)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
       compat.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
       flush_context.execute()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
       rec.execute(self)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
       persistence.save_obj(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
       _emit_insert_statements(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
       result = cached_connections[connection].execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
       return meth(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
       return connection._execute_clauseelement(self, multiparams, params)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
       ret = self._execute_context(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
       self._handle_dbapi_exception(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
       util.raise_(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
       raise exception
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
       self.dialect.do_execute(
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
       cursor.execute(statement, parameters)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
       result = self._query(query)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
       conn.query(q)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
       self._affected_rows = self._read_query_result(unbuffered=unbuffered)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
       result.read()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
       first_packet = self.connection._read_packet()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
       packet.raise_for_error()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
       err.raise_mysql_exception(self._data)
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
       raise errorclass(errno, errval)
   sqlalchemy.exc.ProgrammingError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
   (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '}, 1, 1)' at line 1")
   [SQL: INSERT INTO table_columns (uuid, created_on, changed_on, column_name, verbose_name, is_active, type, groupby, filterable, description, table_id, is_dttm, expression, python_date_format, extra, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(column_name)s, %(verbose_name)s, %(is_active)s, %(type)s, %(groupby)s, %(filterable)s, %(description)s, %(table_id)s, %(is_dttm)s, %(expression)s, %(python_date_format)s, %(extra)s, %(created_by_fk)s, %(changed_by_fk)s)]
   [parameters: {'uuid': b'\xdc\x83\xcbZ=\xdfI\n\xa3\xafv\xad\x8c\x89\x9bE', 'created_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241081), 'changed_on': datetime.datetime(2022, 3, 23, 17, 56, 50, 241092), 'column_name': 'LAST_ACCESS_DATE', 'verbose_name': None, 'is_active': 1, 'type': 'DATETIME', 'groupby': 1, 'filterable': 1, 'description': None, 'table_id': 5, 'is_dttm': 1, 'expression': 'from_unixtime(LAST_ACCESS/1000, "%Y/%m/%d")', 'python_date_format': '%Y/%m/%d', 'extra': {}, 'created_by_fk': 1, 'changed_by_fk': 1}]
   (Background on this error at: http://sqlalche.me/e/13/f405)
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/dashboards/commands/importers/dispatcher.py", line 57, in run
       command.run()
     File "/Users/srinivasan/SourceCode/superset/venv/lib/python3.8/site-packages/superset/commands/importers/v1/__init__.py", line 73, in run
       raise self.import_error() from ex
   superset.dashboards.commands.exceptions.DashboardImportError: Import dashboard failed for an unknown reason
   Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,271:WARNING:superset.views.base:Import dashboard failed for an unknown reason
   2022-03-23 17:56:50,272:INFO:werkzeug:127.0.0.1 - - [23/Mar/2022 17:56:50] "POST /api/v1/dashboard/import/ HTTP/1.1" 500 -
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinisubramanian commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
srinisubramanian commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1075183348


   OK .. more progress and looking good.
   
   Based on comments in the other referenced issue (above), I replaced the code in datasets/commands/export.py from the master branch and it works!!  So looks like solution is already in master and this does not work in 1.4.1 only
   
   old code:
   ```
           for metric in payload.get("metrics", []):
                if metric.get("extra"):
                    try:
                        metric["extra"] = json.loads(metric["extra"])
                    except json.decoder.JSONDecodeError:
                        logger.info("Unable to decode `extra` field: %s", metric["extra"])
   
   ```
   replaced with (from master):
   ```
           for key in ("metrics", "columns"):
               for attributes in payload.get(key, []):
                   if attributes.get("extra"):
                       try:
                           attributes["extra"] = json.loads(attributes["extra"])
                       except json.decoder.JSONDecodeError:
                           logger.info(
                               "Unable to decode `extra` field: %s", attributes["extra"]
                           )
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] betodealmeida commented on issue #19297: Export / Import fails with validation error

Posted by GitBox <gi...@apache.org>.
betodealmeida commented on issue #19297:
URL: https://github.com/apache/superset/issues/19297#issuecomment-1079140117


   Taking a look, sorry for the delay.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org