You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/11/25 02:42:16 UTC
[GitHub] [superset] sairamkrish opened a new issue #17551: Unable to get started with SQLite as Superset metadata database
sairamkrish opened a new issue #17551:
URL: https://github.com/apache/superset/issues/17551
[This thread](https://github.com/apache/superset/issues/8874) says:
> support for SQLite is not removed immediately. But in future it will be removed
Current [Apache superset documentation](https://superset.apache.org/docs/installation/installing-superset-from-scratch) still has SQLite as the default metadata database.
But in reality, Apache superset already doesn't work with SQLite. We can't get started with Apache Superset + SQLite as metadata store today (2021-11-25). Using Apache superset version 1.3.2.
```sh
(superset-experiment) ➜ superset-experiment git:(main) ✗ PYTHONPATH=.:$PYTHONPATH superset db upgrade
Loaded your LOCAL configuration at [/Users/sairam/superset-experiment/superset_config.py]
logging was configured successfully
2021-11-25 07:28:07,681:INFO:superset.utils.logging_configurator:logging was configured successfully
2021-11-25 07:28:07,695:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/flask_caching/__init__.py:201: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
warnings.warn(
No PIL installation found
2021-11-25 07:28:07,915:INFO:superset.utils.screenshots:No PIL installation found
WARNI [alembic.env] SQLite Database support for metadata databases will be removed in a future version of Superset.
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 12d55656cbca -> 2591d77e9831, user_id
Traceback (most recent call last):
File "/Users/sairam/superset-experiment/.venv/bin/superset", line 8, in <module>
sys.exit(superset())
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/flask/cli.py", line 426, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/flask_migrate/cli.py", line 149, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 98, in wrapped
f(*args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 185, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/alembic/command.py", line 320, in upgrade
script.run_env()
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/alembic/script/base.py", line 563, in run_env
util.load_python_file(self.dir, "env.py")
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
module = load_module_py(module_id, path)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/superset/migrations/env.py", line 124, in <module>
run_migrations_online()
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/superset/migrations/env.py", line 116, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
self.get_context().run_migrations(**kw)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
step.migration_fn(**kw)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/superset/migrations/versions/2591d77e9831_user_id.py", line 36, in upgrade
batch_op.create_foreign_key("user_id", "ab_user", ["user_id"], ["id"])
File "/Users/sairam/.pyenv/versions/3.8.1/lib/python3.8/contextlib.py", line 120, in __exit__
next(self.gen)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/alembic/operations/base.py", line 374, in batch_alter_table
impl.flush()
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/alembic/operations/batch.py", line 118, in flush
existing_table = Table(
File "<string>", line 2, in __new__
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 139, in warned
return fn(*args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 563, in __new__
metadata._remove_table(name, schema)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 558, in __new__
table._init(name, metadata, *args, **kw)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 647, in _init
self._autoload(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 670, in _autoload
autoload_with.run_callable(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1653, in run_callable
return callable_(self, *args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 484, in reflecttable
return insp.reflecttable(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 684, in reflecttable
self._reflect_fk(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 868, in _reflect_fk
sa_schema.Table(
File "<string>", line 2, in __new__
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 139, in warned
return fn(*args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 563, in __new__
metadata._remove_table(name, schema)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 558, in __new__
table._init(name, metadata, *args, **kw)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 647, in _init
self._autoload(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 670, in _autoload
autoload_with.run_callable(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1653, in run_callable
return callable_(self, *args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 484, in reflecttable
return insp.reflecttable(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 684, in reflecttable
self._reflect_fk(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 868, in _reflect_fk
sa_schema.Table(
File "<string>", line 2, in __new__
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 139, in warned
return fn(*args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 563, in __new__
metadata._remove_table(name, schema)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 558, in __new__
table._init(name, metadata, *args, **kw)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 647, in _init
self._autoload(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 670, in _autoload
autoload_with.run_callable(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1653, in run_callable
return callable_(self, *args, **kwargs)
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 484, in reflecttable
return insp.reflecttable(
File "/Users/sairam/superset-experiment/.venv/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 678, in reflecttable
raise exc.NoSuchTableError(table.name)
sqlalchemy.exc.NoSuchTableError: ab_user
```
Most amazing feature about Apache Superset is that it was so good to get started quickly. SQLite as metadata store was helping on this.
In production, if someone uses SQLite, I believe that is their responsibility.
Drawing parallels, if someone in production, has Superset admin password as `password` or some other easy to hack password, are we going to remove the support for all common hackable passwords for Apache Superset ?
For small, simple application - SQLite was super useful as a Superset Metadata store. Many applications start small and grow big. I use Superset with SQLite as metadata database in all my data science personal projects, experimental applications etc till the application proves it's existence.
As @xinbinhuang pointed out, Alembic supports SQLite & non SQLite flows. There is a small additional step that we need to do while writing database migration scripts.
I tried finding alternative to SQLite. In Python + SQLAlchemy world, there is no good alternative to SQLite, as far as I know. H2 & HyperSQL are java based and provide only jdbc drivers. SQLAlchemy python+jdbc driver dialects are not mature enough even in November 2021.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida edited a comment on issue #17551: Unable to get started with SQLite as Superset metadata database with custom superset_config.py
Posted by GitBox <gi...@apache.org>.
betodealmeida edited a comment on issue #17551:
URL: https://github.com/apache/superset/issues/17551#issuecomment-988857198
@sairamkrish can you try setting `PREVENT_UNSAFE_DB_CONNECTIONS` to `False` in your `superset_config.py`? That should make it work. I'll take a look into this, I think there's value in Sqlite so I bring the issue up with the community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] betodealmeida commented on issue #17551: Unable to get started with SQLite as Superset metadata database with custom superset_config.py
Posted by GitBox <gi...@apache.org>.
betodealmeida commented on issue #17551:
URL: https://github.com/apache/superset/issues/17551#issuecomment-988857198
@sairamkrish can you try setting `PREVENT_UNSAFE_DB_CONNECTIONS` to `True` in your `superset_config.py`? That should make it work. I'll take a look into this, I think there's value in Sqlite so I bring the issue up with the community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] amitmiran137 closed issue #17551: Unable to get started with SQLite as Superset metadata database with custom superset_config.py
Posted by GitBox <gi...@apache.org>.
amitmiran137 closed issue #17551:
URL: https://github.com/apache/superset/issues/17551
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org