You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "lozbrown (via GitHub)" <gi...@apache.org> on 2023/04/24 15:21:35 UTC
[GitHub] [superset] lozbrown opened a new issue, #23799: Import database fails
lozbrown opened a new issue, #23799:
URL: https://github.com/apache/superset/issues/23799
A functional and working database that was configured via the UI and exported cannot be imported to superset via the cli
#### How to reproduce the bug
Use superset import_datasources -p to import
### Expected results
Database successfully imported and updated
### Actual results
> `root@superset-7759bb8887-sj9ln:/app# superset import_datasources -p /app/import_datasources_fmt.yaml
> Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
> logging was configured successfully
> 2023-04-24 15:04:46,625:INFO:superset.utils.logging_configurator:logging was configured successfully
> 2023-04-24 15:04:46,630:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
> We haven't found any Content Security Policy (CSP) defined in the configurations. Please make sure to configure CSP using the TALISMAN_CONFIG key or any other external software. Failing to configure CSP have serious security implications. Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key.
> 2023-04-24 15:04:46,631:WARNING:superset.initialization:We haven't found any Content Security Policy (CSP) defined in the configurations. Please make sure to configure CSP using the TALISMAN_CONFIG key or any other external software. Failing to configure CSP have serious security implications. Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key.
> /usr/local/lib/python3.8/site-packages/flask_caching/__init__.py:201: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
> warnings.warn(
> Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
> 2023-04-24 15:04:46,633:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
> Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
> 2023-04-24 15:04:46,635:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
> Importing dataset from file /app/import_datasources_fmt.yaml
> 2023-04-24 15:04:48,007:INFO:superset.datasets.commands.importers.v0:Importing dataset from file /app/import_datasources_fmt.yaml
> Importing 1 databases
> 2023-04-24 15:04:48,007:INFO:superset.datasets.commands.importers.v0:Importing 1 databases
> Importing new dbs prestodev2
> 2023-04-24 15:04:48,025:INFO:superset.models.helpers:Importing new dbs prestodev2
> Error running import command
> Traceback (most recent call last):
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
> self.dialect.do_execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
> cursor.execute(statement, parameters)
> psycopg2.ProgrammingError: can't adapt type 'dict'
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File "/app/superset/datasets/commands/importers/dispatcher.py", line 57, in run
> command.run()
> File "/app/superset/datasets/commands/importers/v0.py", line 261, in run
> import_from_dict(db.session, config, sync=self.sync)
> File "/app/superset/datasets/commands/importers/v0.py", line 225, in import_from_dict
> session.commit()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/scoping.py", line 163, in do
> return getattr(self.registry(), name)(*args, **kwargs)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1046, in commit
> self.transaction.commit()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 504, in commit
> self._prepare_impl()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 483, in _prepare_impl
> self.session.flush()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
> self._flush(objects)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
> transaction.rollback(_capture_exception=True)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
> compat.raise_(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
> raise exception
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
> flush_context.execute()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
> rec.execute(self)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
> persistence.save_obj(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
> _emit_insert_statements(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
> result = cached_connections[connection].execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
> return meth(self, multiparams, params)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
> ret = self._execute_context(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
> self._handle_dbapi_exception(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
> util.raise_(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
> raise exception
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
> self.dialect.do_execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
> cursor.execute(statement, parameters)
> sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'dict'
> [SQL: INSERT INTO dbs (uuid, created_on, changed_on, verbose_name, database_name, sqlalchemy_uri, password, cache_timeout, select_as_create_table_as, expose_in_sqllab, allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml, force_ctas_schema, allow_multi_schema_metadata_fetch, extra, encrypted_extra, impersonate_user, server_cert, is_managed_externally, external_url, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(verbose_name)s, %(database_name)s, %(sqlalchemy_uri)s, %(password)s, %(cache_timeout)s, %(select_as_create_table_as)s, %(expose_in_sqllab)s, %(allow_run_async)s, %(allow_file_upload)s, %(allow_ctas)s, %(allow_cvas)s, %(allow_dml)s, %(force_ctas_schema)s, %(allow_multi_schema_metadata_fetch)s, %(extra)s, %(encrypted_extra)s, %(impersonate_user)s, %(server_cert)s, %(is_managed_externally)s, %(external_url)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING dbs.id]
> [parameters: {'uuid': UUID('48bc9058-3e74-48d4-85ef-8b94f5044165'), 'created_on': datetime.datetime(2023, 4, 24, 15, 4, 48, 27077), 'changed_on': datetime.datetime(2023, 4, 24, 15, 4, 48, 27082), 'verbose_name': None, 'database_name': 'prestodev2', 'sqlalchemy_uri': "presto://MYDEVUSER:MYDEVPASSWORD@quasar-query-dev.macbank:4039", 'password': <psycopg2.extensions.Binary object at 0x7efd2d8a0ed0>, 'cache_timeout': None, 'select_as_create_table_as': False, 'expose_in_sqllab': True, 'allow_run_async': False, 'allow_file_upload': False, 'allow_ctas': False, 'allow_cvas': True, 'allow_dml': False, 'force_ctas_schema': None, 'allow_multi_schema_metadata_fetch': False, 'extra': {'engine_params': {'connect_args': {'protocol': 'https', 'source': 'superset 2.0.1', 'requests_kwargs': {'verify': '/app/mac-ca-bundle'}}}, 'metadata_params': {}, 'schemas_allowed_for_csv_upload': []}, 'encrypted_extra': None, 'impersonate_user': False, 'server_cert': None, 'is_managed_externally': False, 'exter
nal_url': None, 'created_by_fk': None, 'changed_by_fk': None}]
> (Background on this error at: http://sqlalche.me/e/13/f405)
> 2023-04-24 15:04:48,028:ERROR:superset.datasets.commands.importers.dispatcher:Error running import command
> Traceback (most recent call last):
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
> self.dialect.do_execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
> cursor.execute(statement, parameters)
> psycopg2.ProgrammingError: can't adapt type 'dict'
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File "/app/superset/datasets/commands/importers/dispatcher.py", line 57, in run
> command.run()
> File "/app/superset/datasets/commands/importers/v0.py", line 261, in run
> import_from_dict(db.session, config, sync=self.sync)
> File "/app/superset/datasets/commands/importers/v0.py", line 225, in import_from_dict
> session.commit()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/scoping.py", line 163, in do
> return getattr(self.registry(), name)(*args, **kwargs)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1046, in commit
> self.transaction.commit()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 504, in commit
> self._prepare_impl()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 483, in _prepare_impl
> self.session.flush()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
> self._flush(objects)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
> transaction.rollback(_capture_exception=True)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
> compat.raise_(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
> raise exception
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
> flush_context.execute()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
> rec.execute(self)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
> persistence.save_obj(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
> _emit_insert_statements(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
> result = cached_connections[connection].execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
> return meth(self, multiparams, params)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
> ret = self._execute_context(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
> self._handle_dbapi_exception(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
> util.raise_(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
> raise exception
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
> self.dialect.do_execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
> cursor.execute(statement, parameters)
> sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'dict'
> [SQL: INSERT INTO dbs (uuid, created_on, changed_on, verbose_name, database_name, sqlalchemy_uri, password, cache_timeout, select_as_create_table_as, expose_in_sqllab, allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml, force_ctas_schema, allow_multi_schema_metadata_fetch, extra, encrypted_extra, impersonate_user, server_cert, is_managed_externally, external_url, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(verbose_name)s, %(database_name)s, %(sqlalchemy_uri)s, %(password)s, %(cache_timeout)s, %(select_as_create_table_as)s, %(expose_in_sqllab)s, %(allow_run_async)s, %(allow_file_upload)s, %(allow_ctas)s, %(allow_cvas)s, %(allow_dml)s, %(force_ctas_schema)s, %(allow_multi_schema_metadata_fetch)s, %(extra)s, %(encrypted_extra)s, %(impersonate_user)s, %(server_cert)s, %(is_managed_externally)s, %(external_url)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING dbs.id]
> [parameters: {'uuid': UUID('48bc9058-3e74-48d4-85ef-8b94f5044165'), 'created_on': datetime.datetime(2023, 4, 24, 15, 4, 48, 27077), 'changed_on': datetime.datetime(2023, 4, 24, 15, 4, 48, 27082), 'verbose_name': None, 'database_name': 'prestodev2', 'sqlalchemy_uri': "presto://MYDEVUSER:MYDEVPASSWORD@quasar-query-dev.macbank:4039", 'password': <psycopg2.extensions.Binary object at 0x7efd2d8a0ed0>, 'cache_timeout': None, 'select_as_create_table_as': False, 'expose_in_sqllab': True, 'allow_run_async': False, 'allow_file_upload': False, 'allow_ctas': False, 'allow_cvas': True, 'allow_dml': False, 'force_ctas_schema': None, 'allow_multi_schema_metadata_fetch': False, 'extra': {'engine_params': {'connect_args': {'protocol': 'https', 'source': 'superset 2.0.1', 'requests_kwargs': {'verify': '/app/mac-ca-bundle'}}}, 'metadata_params': {}, 'schemas_allowed_for_csv_upload': []}, 'encrypted_extra': None, 'impersonate_user': False, 'server_cert': None, 'is_managed_externally': False, 'exter
nal_url': None, 'created_by_fk': None, 'changed_by_fk': None}]
> (Background on this error at: http://sqlalche.me/e/13/f405)
> There was an error when importing the dataset(s), please check the exception traceback in the log
> Traceback (most recent call last):
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
> self.dialect.do_execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
> cursor.execute(statement, parameters)
> psycopg2.ProgrammingError: can't adapt type 'dict'
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File "/app/superset/cli/importexport.py", line 193, in import_datasources
> ImportDatasetsCommand(contents, overwrite=True).run()
> File "/app/superset/datasets/commands/importers/dispatcher.py", line 68, in run
> raise exc
> File "/app/superset/datasets/commands/importers/dispatcher.py", line 57, in run
> command.run()
> File "/app/superset/datasets/commands/importers/v0.py", line 261, in run
> import_from_dict(db.session, config, sync=self.sync)
> File "/app/superset/datasets/commands/importers/v0.py", line 225, in import_from_dict
> session.commit()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/scoping.py", line 163, in do
> return getattr(self.registry(), name)(*args, **kwargs)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1046, in commit
> self.transaction.commit()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 504, in commit
> self._prepare_impl()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 483, in _prepare_impl
> self.session.flush()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
> self._flush(objects)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
> transaction.rollback(_capture_exception=True)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
> compat.raise_(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
> raise exception
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
> flush_context.execute()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
> rec.execute(self)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
> persistence.save_obj(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
> _emit_insert_statements(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
> result = cached_connections[connection].execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
> return meth(self, multiparams, params)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
> ret = self._execute_context(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
> self._handle_dbapi_exception(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
> util.raise_(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
> raise exception
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
> self.dialect.do_execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
> cursor.execute(statement, parameters)
> sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'dict'
> [SQL: INSERT INTO dbs (uuid, created_on, changed_on, verbose_name, database_name, sqlalchemy_uri, password, cache_timeout, select_as_create_table_as, expose_in_sqllab, allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml, force_ctas_schema, allow_multi_schema_metadata_fetch, extra, encrypted_extra, impersonate_user, server_cert, is_managed_externally, external_url, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(verbose_name)s, %(database_name)s, %(sqlalchemy_uri)s, %(password)s, %(cache_timeout)s, %(select_as_create_table_as)s, %(expose_in_sqllab)s, %(allow_run_async)s, %(allow_file_upload)s, %(allow_ctas)s, %(allow_cvas)s, %(allow_dml)s, %(force_ctas_schema)s, %(allow_multi_schema_metadata_fetch)s, %(extra)s, %(encrypted_extra)s, %(impersonate_user)s, %(server_cert)s, %(is_managed_externally)s, %(external_url)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING dbs.id]
> [parameters: {'uuid': UUID('48bc9058-3e74-48d4-85ef-8b94f5044165'), 'created_on': datetime.datetime(2023, 4, 24, 15, 4, 48, 27077), 'changed_on': datetime.datetime(2023, 4, 24, 15, 4, 48, 27082), 'verbose_name': None, 'database_name': 'prestodev2', 'sqlalchemy_uri': "presto://MYDEVUSER:MYDEVPASSWORD@quasar-query-dev.macbank:4039", 'password': <psycopg2.extensions.Binary object at 0x7efd2d8a0ed0>, 'cache_timeout': None, 'select_as_create_table_as': False, 'expose_in_sqllab': True, 'allow_run_async': False, 'allow_file_upload': False, 'allow_ctas': False, 'allow_cvas': True, 'allow_dml': False, 'force_ctas_schema': None, 'allow_multi_schema_metadata_fetch': False, 'extra': {'engine_params': {'connect_args': {'protocol': 'https', 'source': 'superset 2.0.1', 'requests_kwargs': {'verify': '/app/mac-ca-bundle'}}}, 'metadata_params': {}, 'schemas_allowed_for_csv_upload': []}, 'encrypted_extra': None, 'impersonate_user': False, 'server_cert': None, 'is_managed_externally': False, 'exter
nal_url': None, 'created_by_fk': None, 'changed_by_fk': None}]
> (Background on this error at: http://sqlalche.me/e/13/f405)
> 2023-04-24 15:04:48,032:ERROR:superset.cli.importexport:There was an error when importing the dataset(s), please check the exception traceback in the log
> Traceback (most recent call last):
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
> self.dialect.do_execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
> cursor.execute(statement, parameters)
> psycopg2.ProgrammingError: can't adapt type 'dict'
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File "/app/superset/cli/importexport.py", line 193, in import_datasources
> ImportDatasetsCommand(contents, overwrite=True).run()
> File "/app/superset/datasets/commands/importers/dispatcher.py", line 68, in run
> raise exc
> File "/app/superset/datasets/commands/importers/dispatcher.py", line 57, in run
> command.run()
> File "/app/superset/datasets/commands/importers/v0.py", line 261, in run
> import_from_dict(db.session, config, sync=self.sync)
> File "/app/superset/datasets/commands/importers/v0.py", line 225, in import_from_dict
> session.commit()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/scoping.py", line 163, in do
> return getattr(self.registry(), name)(*args, **kwargs)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1046, in commit
> self.transaction.commit()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 504, in commit
> self._prepare_impl()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 483, in _prepare_impl
> self.session.flush()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
> self._flush(objects)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
> transaction.rollback(_capture_exception=True)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
> compat.raise_(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
> raise exception
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
> flush_context.execute()
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
> rec.execute(self)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
> persistence.save_obj(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
> _emit_insert_statements(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
> result = cached_connections[connection].execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
> return meth(self, multiparams, params)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
> return connection._execute_clauseelement(self, multiparams, params)
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
> ret = self._execute_context(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
> self._handle_dbapi_exception(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
> util.raise_(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
> raise exception
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
> self.dialect.do_execute(
> File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
> cursor.execute(statement, parameters)
> sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'dict'
> [SQL: INSERT INTO dbs (uuid, created_on, changed_on, verbose_name, database_name, sqlalchemy_uri, password, cache_timeout, select_as_create_table_as, expose_in_sqllab, allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml, force_ctas_schema, allow_multi_schema_metadata_fetch, extra, encrypted_extra, impersonate_user, server_cert, is_managed_externally, external_url, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(verbose_name)s, %(database_name)s, %(sqlalchemy_uri)s, %(password)s, %(cache_timeout)s, %(select_as_create_table_as)s, %(expose_in_sqllab)s, %(allow_run_async)s, %(allow_file_upload)s, %(allow_ctas)s, %(allow_cvas)s, %(allow_dml)s, %(force_ctas_schema)s, %(allow_multi_schema_metadata_fetch)s, %(extra)s, %(encrypted_extra)s, %(impersonate_user)s, %(server_cert)s, %(is_managed_externally)s, %(external_url)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING dbs.id]
> [parameters: {'uuid': UUID('48bc9058-3e74-48d4-85ef-8b94f5044165'), 'created_on': datetime.datetime(2023, 4, 24, 15, 4, 48, 27077), 'changed_on': datetime.datetime(2023, 4, 24, 15, 4, 48, 27082), 'verbose_name': None, 'database_name': 'prestodev2', 'sqlalchemy_uri': "presto://MYDEVUSER:MYDEVPASSWORD@quasar-query-dev.macbank:4039", 'password': <psycopg2.extensions.Binary object at 0x7efd2d8a0ed0>, 'cache_timeout': None, 'select_as_create_table_as': False, 'expose_in_sqllab': True, 'allow_run_async': False, 'allow_file_upload': False, 'allow_ctas': False, 'allow_cvas': True, 'allow_dml': False, 'force_ctas_schema': None, 'allow_multi_schema_metadata_fetch': False, 'extra': {'engine_params': {'connect_args': {'protocol': 'https', 'source': 'superset 2.0.1', 'requests_kwargs': {'verify': '/app/mac-ca-bundle'}}}, 'metadata_params': {}, 'schemas_allowed_for_csv_upload': []}, 'encrypted_extra': None, 'impersonate_user': False, 'server_cert': None, 'is_managed_externally': False, 'exter
nal_url': None, 'created_by_fk': None, 'changed_by_fk': None}]
> (Background on this error at: http://sqlalche.me/e/13/f405)`
>
### Environment
(please complete the following information):
- superset version: 2.0.1
- python version: Python 3.8.1 inside official docker image
- node.js version: `node -v` /N/A
- any feature flags active:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Import database fails [superset]
Posted by "mike-code (via GitHub)" <gi...@apache.org>.
mike-code commented on issue #23799:
URL: https://github.com/apache/superset/issues/23799#issuecomment-1851083427
I have the same issue. It appears that the importer tries to pass the following json parameter which results in the error
```
'extra': {'allows_virtual_table_explore': True, 'cost_estimate_enabled': True},
```
If I remove the `extra` key from the yaml file I want to import, the database will import successfully.
My import file:
```
databases:
- sqlalchemy_uri: "postgresql://user:password@10.124.0.21:5432/dbname"
database_name: the-db
cache_timeout: null
expose_in_sqllab: true
allow_run_async: false
allow_ctas: false
allow_cvas: true
allow_dml: false
allow_csv_upload: false
extra:
allows_virtual_table_explore: true
cost_estimate_enabled: true
version: 1.0.0
```
<details>
<summary>python exception</summary>
```
Loaded your LOCAL configuration at [/app/pythonpath/superset_config.py]
logging was configured successfully
2023-12-11 23:54:36,761:INFO:superset.utils.logging_configurator:logging was configured successfully
2023-12-11 23:54:36,772:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/usr/local/lib/python3.9/site-packages/flask_limiter/extension.py:293: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
warnings.warn(
Importing dataset from file foo.yaml
2023-12-11 23:54:38,347:INFO:superset.datasets.commands.importers.v0:Importing dataset from file foo.yaml
Importing 1 databases
2023-12-11 23:54:38,348:INFO:superset.datasets.commands.importers.v0:Importing 1 databases
Importing new dbs the-db
2023-12-11 23:54:38,357:INFO:superset.models.helpers:Importing new dbs the-db
Error running import command
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
psycopg2.ProgrammingError: can't adapt type 'dict'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/superset/datasets/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/app/superset/datasets/commands/importers/v0.py", line 261, in run
import_from_dict(db.session, config, sync=self.sync)
File "/app/superset/datasets/commands/importers/v0.py", line 225, in import_from_dict
session.commit()
File "<string>", line 2, in commit
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1435, in commit
self._transaction.commit(_to_root=self.future)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 829, in commit
self._prepare_impl()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3367, in flush
self._flush(objects)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3507, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3467, in _flush
flush_context.execute()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
rec.execute(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
util.preloaded.orm_persistence.save_obj(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
_emit_insert_statements(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements
result = connection._execute_20(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'dict'
[SQL: INSERT INTO dbs (uuid, created_on, changed_on, verbose_name, database_name, sqlalchemy_uri, password, cache_timeout, select_as_create_table_as, expose_in_sqllab, allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml, force_ctas_schema, extra, encrypted_extra, impersonate_user, server_cert, is_managed_externally, external_url, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(verbose_name)s, %(database_name)s, %(sqlalchemy_uri)s, %(password)s, %(cache_timeout)s, %(select_as_create_table_as)s, %(expose_in_sqllab)s, %(allow_run_async)s, %(allow_file_upload)s, %(allow_ctas)s, %(allow_cvas)s, %(allow_dml)s, %(force_ctas_schema)s, %(extra)s, %(encrypted_extra)s, %(impersonate_user)s, %(server_cert)s, %(is_managed_externally)s, %(external_url)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING dbs.id]
[parameters: {'uuid': UUID('f0576fce-20f8-49a1-ae3a-a911408b8783'), 'created_on': datetime.datetime(2023, 12, 11, 23, 54, 38, 361669), 'changed_on': datetime.datetime(2023, 12, 11, 23, 54, 38, 361676), 'verbose_name': None, 'database_name': 'the-db', 'sqlalchemy_uri': 'postgresql://myuser:mypassword@10.124.0.21:5432/dbname', 'password': None, 'cache_timeout': None, 'select_as_create_table_as': False, 'expose_in_sqllab': True, 'allow_run_async': False, 'allow_file_upload': False, 'allow_ctas': False, 'allow_cvas': True, 'allow_dml': False, 'force_ctas_schema': None, 'extra': {'allows_virtual_table_explore': True, 'cost_estimate_enabled': True}, 'encrypted_extra': None, 'impersonate_user': False, 'server_cert': None, 'is_managed_externally': False, 'external_url': None, 'created_by_fk': None, 'changed_by_fk': None}]
(Background on this error at: https://sqlalche.me/e/14/f405)
2023-12-11 23:54:38,363:ERROR:superset.datasets.commands.importers.dispatcher:Error running import command
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
psycopg2.ProgrammingError: can't adapt type 'dict'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/superset/datasets/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/app/superset/datasets/commands/importers/v0.py", line 261, in run
import_from_dict(db.session, config, sync=self.sync)
File "/app/superset/datasets/commands/importers/v0.py", line 225, in import_from_dict
session.commit()
File "<string>", line 2, in commit
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1435, in commit
self._transaction.commit(_to_root=self.future)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 829, in commit
self._prepare_impl()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3367, in flush
self._flush(objects)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3507, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3467, in _flush
flush_context.execute()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
rec.execute(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
util.preloaded.orm_persistence.save_obj(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
_emit_insert_statements(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements
result = connection._execute_20(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'dict'
[SQL: INSERT INTO dbs (uuid, created_on, changed_on, verbose_name, database_name, sqlalchemy_uri, password, cache_timeout, select_as_create_table_as, expose_in_sqllab, allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml, force_ctas_schema, extra, encrypted_extra, impersonate_user, server_cert, is_managed_externally, external_url, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(verbose_name)s, %(database_name)s, %(sqlalchemy_uri)s, %(password)s, %(cache_timeout)s, %(select_as_create_table_as)s, %(expose_in_sqllab)s, %(allow_run_async)s, %(allow_file_upload)s, %(allow_ctas)s, %(allow_cvas)s, %(allow_dml)s, %(force_ctas_schema)s, %(extra)s, %(encrypted_extra)s, %(impersonate_user)s, %(server_cert)s, %(is_managed_externally)s, %(external_url)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING dbs.id]
[parameters: {'uuid': UUID('f0576fce-20f8-49a1-ae3a-a911408b8783'), 'created_on': datetime.datetime(2023, 12, 11, 23, 54, 38, 361669), 'changed_on': datetime.datetime(2023, 12, 11, 23, 54, 38, 361676), 'verbose_name': None, 'database_name': 'the-db', 'sqlalchemy_uri': 'postgresql://myuser:mypassword@10.124.0.21:5432/dbname', 'password': None, 'cache_timeout': None, 'select_as_create_table_as': False, 'expose_in_sqllab': True, 'allow_run_async': False, 'allow_file_upload': False, 'allow_ctas': False, 'allow_cvas': True, 'allow_dml': False, 'force_ctas_schema': None, 'extra': {'allows_virtual_table_explore': True, 'cost_estimate_enabled': True}, 'encrypted_extra': None, 'impersonate_user': False, 'server_cert': None, 'is_managed_externally': False, 'external_url': None, 'created_by_fk': None, 'changed_by_fk': None}]
(Background on this error at: https://sqlalche.me/e/14/f405)
There was an error when importing the dataset(s), please check the exception traceback in the log
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
psycopg2.ProgrammingError: can't adapt type 'dict'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/superset/cli/importexport.py", line 194, in import_datasources
ImportDatasetsCommand(contents, overwrite=True).run()
File "/app/superset/datasets/commands/importers/dispatcher.py", line 68, in run
raise exc
File "/app/superset/datasets/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/app/superset/datasets/commands/importers/v0.py", line 261, in run
import_from_dict(db.session, config, sync=self.sync)
File "/app/superset/datasets/commands/importers/v0.py", line 225, in import_from_dict
session.commit()
File "<string>", line 2, in commit
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1435, in commit
self._transaction.commit(_to_root=self.future)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 829, in commit
self._prepare_impl()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3367, in flush
self._flush(objects)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3507, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3467, in _flush
flush_context.execute()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
rec.execute(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
util.preloaded.orm_persistence.save_obj(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
_emit_insert_statements(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements
result = connection._execute_20(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'dict'
[SQL: INSERT INTO dbs (uuid, created_on, changed_on, verbose_name, database_name, sqlalchemy_uri, password, cache_timeout, select_as_create_table_as, expose_in_sqllab, allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml, force_ctas_schema, extra, encrypted_extra, impersonate_user, server_cert, is_managed_externally, external_url, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(verbose_name)s, %(database_name)s, %(sqlalchemy_uri)s, %(password)s, %(cache_timeout)s, %(select_as_create_table_as)s, %(expose_in_sqllab)s, %(allow_run_async)s, %(allow_file_upload)s, %(allow_ctas)s, %(allow_cvas)s, %(allow_dml)s, %(force_ctas_schema)s, %(extra)s, %(encrypted_extra)s, %(impersonate_user)s, %(server_cert)s, %(is_managed_externally)s, %(external_url)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING dbs.id]
[parameters: {'uuid': UUID('f0576fce-20f8-49a1-ae3a-a911408b8783'), 'created_on': datetime.datetime(2023, 12, 11, 23, 54, 38, 361669), 'changed_on': datetime.datetime(2023, 12, 11, 23, 54, 38, 361676), 'verbose_name': None, 'database_name': 'the-db', 'sqlalchemy_uri': 'postgresql://myuser:mypassword@10.124.0.21:5432/dbname', 'password': None, 'cache_timeout': None, 'select_as_create_table_as': False, 'expose_in_sqllab': True, 'allow_run_async': False, 'allow_file_upload': False, 'allow_ctas': False, 'allow_cvas': True, 'allow_dml': False, 'force_ctas_schema': None, 'extra': {'allows_virtual_table_explore': True, 'cost_estimate_enabled': True}, 'encrypted_extra': None, 'impersonate_user': False, 'server_cert': None, 'is_managed_externally': False, 'external_url': None, 'created_by_fk': None, 'changed_by_fk': None}]
(Background on this error at: https://sqlalche.me/e/14/f405)
2023-12-11 23:54:38,368:ERROR:superset.cli.importexport:There was an error when importing the dataset(s), please check the exception traceback in the log
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
psycopg2.ProgrammingError: can't adapt type 'dict'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/superset/cli/importexport.py", line 194, in import_datasources
ImportDatasetsCommand(contents, overwrite=True).run()
File "/app/superset/datasets/commands/importers/dispatcher.py", line 68, in run
raise exc
File "/app/superset/datasets/commands/importers/dispatcher.py", line 57, in run
command.run()
File "/app/superset/datasets/commands/importers/v0.py", line 261, in run
import_from_dict(db.session, config, sync=self.sync)
File "/app/superset/datasets/commands/importers/v0.py", line 225, in import_from_dict
session.commit()
File "<string>", line 2, in commit
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1435, in commit
self._transaction.commit(_to_root=self.future)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 829, in commit
self._prepare_impl()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3367, in flush
self._flush(objects)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3507, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3467, in _flush
flush_context.execute()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
rec.execute(self)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
util.preloaded.orm_persistence.save_obj(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
_emit_insert_statements(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements
result = connection._execute_20(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'dict'
[SQL: INSERT INTO dbs (uuid, created_on, changed_on, verbose_name, database_name, sqlalchemy_uri, password, cache_timeout, select_as_create_table_as, expose_in_sqllab, allow_run_async, allow_file_upload, allow_ctas, allow_cvas, allow_dml, force_ctas_schema, extra, encrypted_extra, impersonate_user, server_cert, is_managed_externally, external_url, created_by_fk, changed_by_fk) VALUES (%(uuid)s, %(created_on)s, %(changed_on)s, %(verbose_name)s, %(database_name)s, %(sqlalchemy_uri)s, %(password)s, %(cache_timeout)s, %(select_as_create_table_as)s, %(expose_in_sqllab)s, %(allow_run_async)s, %(allow_file_upload)s, %(allow_ctas)s, %(allow_cvas)s, %(allow_dml)s, %(force_ctas_schema)s, %(extra)s, %(encrypted_extra)s, %(impersonate_user)s, %(server_cert)s, %(is_managed_externally)s, %(external_url)s, %(created_by_fk)s, %(changed_by_fk)s) RETURNING dbs.id]
[parameters: {'uuid': UUID('f0576fce-20f8-49a1-ae3a-a911408b8783'), 'created_on': datetime.datetime(2023, 12, 11, 23, 54, 38, 361669), 'changed_on': datetime.datetime(2023, 12, 11, 23, 54, 38, 361676), 'verbose_name': None, 'database_name': 'the-db', 'sqlalchemy_uri': 'postgresql://myuser:mypassword@10.124.0.21:5432/dbname', 'password': None, 'cache_timeout': None, 'select_as_create_table_as': False, 'expose_in_sqllab': True, 'allow_run_async': False, 'allow_file_upload': False, 'allow_ctas': False, 'allow_cvas': True, 'allow_dml': False, 'force_ctas_schema': None, 'extra': {'allows_virtual_table_explore': True, 'cost_estimate_enabled': True}, 'encrypted_extra': None, 'impersonate_user': False, 'server_cert': None, 'is_managed_externally': False, 'external_url': None, 'created_by_fk': None, 'changed_by_fk': None}]
(Background on this error at: https://sqlalche.me/e/14/f405)
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Import database fails [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #23799:
URL: https://github.com/apache/superset/issues/23799#issuecomment-2073611156
I'm not sure if this is still an issue in current versions of Superset (3.x). If it is, we can re-open this, or feel free to open a new issue with updated context and a reproducible case using example data. We're no longer supporting Superset 2.x or prior, and it's been a while since this thread saw any activity, so I'm closing this as stale.
Not sure if @betodealmeida sees any action worth taking on this, but I'll leave the ping just in case :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Import database fails [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #23799: Import database fails
URL: https://github.com/apache/superset/issues/23799
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org