You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "StormingBullet (via GitHub)" <gi...@apache.org> on 2023/04/07 11:51:11 UTC
[GitHub] [superset] StormingBullet opened a new issue, #23622: Failed run with 'superset db upgrade'
StormingBullet opened a new issue, #23622:
URL: https://github.com/apache/superset/issues/23622
### Environment
- superset version: 2.1.0
- python version: (source-compiled)3.8.16 (and same issue with 3.10.11)
- OS: CentOS 7.7 (almost but not completly vanilla)
- DBMS: no. just sqlite (for test)
#### How to reproduce the bug
※ First, this server will be used only for superset so it will not use python's virtualized environments.
1. install needed OS package (root user)
yum install gcc gcc-c++ libffi-devel openssl-devel cyrus-sasl-devel openldap-devel
yum install bzip2-devel sqlite-devel
2. compile python (root user)
- ./configure --with-openssl=/usr/local/ssl (it's source-compiled openssl 1.1.1g's path)
- make
- make test
- make install
3. replace python's original symbolic link with python3 (Not directly related to this issue) (root user)
this server will be use for dedicated superset server so no need python2 almost. (/usr/local/bin is server's first path in OS environment)
cd /usr/bin
rm python
ln -s /usr/local/bin/python3 python
cd /usr/local/bin
mv pip pip2 # original 'pip' was linked to pip2.6
ln -s /usr/local/bin/pip3 pip
4. edit yum's files (Not directly related to this issue) (root user)
edit first line of below files from '#!/usr/bin/python' to '#!/usr/bin/python2' for use of yum.
/usr/bin/yum
/usr/libexec/urlgrabber-ext-down
5. install needed python package (root user)
pip install wheel Pillow
6. install superset (still, root user for global install)
pip install apache-superset
7. setting environment for superset (normal user)
add below to content to user's .bash_profile
```
export FLASK_APP=superset
export SUPERSET_CONFIG_PATH=~/.superset/superset_config.py
```
then mkdir ~/.superset for config file location.
run 'openssl rand -base64 42' for secret key.
create '~/.superset/superset_config.py' file and input a line.
`SECRET_KEY = 'OPENSSL_RANDOM_VALUE`
8. initialize the database (normal user)
run 'superset db upgrade'
### Expected results
just complete initialize the database
### Actual results
```
$ superset db upgrade
Loaded your LOCAL configuration at [/home/secuser/.superset/superset_config.py]
logging was configured successfully
2023-04-07 18:49:05,225:INFO:superset.utils.logging_configurator:logging was configured successfully
2023-04-07 18:49:05,231:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
We haven't found any Content Security Policy (CSP) defined in the configurations. Please make sure to configure CSP using the TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other external software. Failing to configure CSP have serious security implications. Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key.
2023-04-07 18:49:05,231:WARNING:superset.initialization:We haven't found any Content Security Policy (CSP) defined in the configurations. Please make sure to configure CSP using the TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other external software. Failing to configure CSP have serious security implications. Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key.
Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
2023-04-07 18:49:05,234:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
2023-04-07 18:49:05,236:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
/home/secuser/.local/lib/python3.8/site-packages/flask_appbuilder/models/sqla/interface.py:67: SAWarning: relationship 'SqlaTable.slices' will copy column tables.id to column slices.datasource_id, which conflicts with relationship(s): 'Slice.table' (copies tables.id to slices.datasource_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="table"' to the 'SqlaTable.slices' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
for prop in class_mapper(obj).iterate_properties:
WARNI [alembic.env] SQLite Database support for metadata databases will be removed in a future version of Superset.
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
**(Omit success info by previous runs)**
INFO [alembic.runtime.migration] Running upgrade 409c7b420ab0 -> ffa79af61a56, rename report_schedule.extra to extra_json
Traceback (most recent call last):
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: near "extra": syntax error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/superset", line 8, in <module>
sys.exit(superset())
File "/home/secuser/.local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 567, in main
return super().main(*args, **kwargs)
File "/home/secuser/.local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/secuser/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/secuser/.local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/secuser/.local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/secuser/.local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/secuser/.local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 407, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/home/secuser/.local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_migrate/cli.py", line 149, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "/usr/local/lib/python3.8/site-packages/flask_migrate/__init__.py", line 98, in wrapped
f(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_migrate/__init__.py", line 185, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/command.py", line 294, in upgrade
script.run_env()
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/script/base.py", line 490, in run_env
util.load_python_file(self.dir, "env.py")
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
module = load_module_py(module_id, path)
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 843, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.8/site-packages/superset/extensions/../migrations/env.py", line 126, in <module>
run_migrations_online()
File "/usr/local/lib/python3.8/site-packages/superset/extensions/../migrations/env.py", line 118, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
self.get_context().run_migrations(**kw)
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 561, in run_migrations
step.migration_fn(**kw)
File "/usr/local/lib/python3.8/site-packages/superset/migrations/versions/2022-07-11_11-26_ffa79af61a56_rename_report_schedule_extra_to_extra_.py", line 36, in upgrade
op.alter_column(
File "<string>", line 8, in alter_column
File "<string>", line 3, in alter_column
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/operations/ops.py", line 1655, in alter_column
return operations.invoke(alt)
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/operations/base.py", line 354, in invoke
return fn(self, operation)
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/operations/toimpl.py", line 43, in alter_column
operations.impl.alter_column(
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 243, in alter_column
self._exec(
File "/home/secuser/.local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 146, in _exec
return conn.execute(construct, multiparams)
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1306, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
return connection._execute_ddl(
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1398, in _execute_ddl
ret = self._execute_context(
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
self._handle_dbapi_exception(
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
util.raise_(
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/home/secuser/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
**sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "extra": syntax error
[SQL: ALTER TABLE report_schedule RENAME extra TO extra_json]**
(Background on this error at: https://sqlalche.me/e/14/e3q8)
```
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [ X ] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [ X ] I have reproduced the issue with at least the latest released version of superset.
- [ X ] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
(1) There is no dependency warning between first and last.
(2) I have checked 'https://github.com/apache/superset/blob/2.1/requirements/base.txt' and all module's version meet conditions.
(3) There seemed to be no problem until the last command was executed.
(4) This issue did not occur in more recent OS (the latest Amazon linux in my case).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] StormingBullet commented on issue #23622: Failed run with 'superset db upgrade'
Posted by "StormingBullet (via GitHub)" <gi...@apache.org>.
StormingBullet commented on issue #23622:
URL: https://github.com/apache/superset/issues/23622#issuecomment-1506333949
> @StormingBullet I had same issue. After multiple tries, I got solution.
>
> it was Sqlite version issue. Actually centos7 comes with very old version of Sqlite. I have replaced Sqlite with postgres 15 `SQLALCHEMY_DATABASE_URI = 'postgresql://postgres:postgres@localhost/postgres' ` and it worked fine.
Thank you for your answer.
However, it seems that it is not just a problem with the old version of sqlite3.
I compiled and installed sqlite3 to the latest version and tried again, but the result was the same.
(Of course, the symbolic link has been changed accordingly.)
```
$ sqlite3 --version
3.41.2 2023-03-22 11:56:21 0d1fc92f94cb6b76bffe3ec34d69cffde2924203304e8ffc4155597af0c191da
```
But there was no difference.
Anyway, as you said, this problem was solved when using postgres. (But another error related to the certificate appeared during the sample data generation process. maybe it's not related to postgres.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] masayyed commented on issue #23622: Failed run with 'superset db upgrade'
Posted by "masayyed (via GitHub)" <gi...@apache.org>.
masayyed commented on issue #23622:
URL: https://github.com/apache/superset/issues/23622#issuecomment-1507093762
Is it related to self signed certificate?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Sumitkumar1007 commented on issue #23622: Failed run with 'superset db upgrade'
Posted by "Sumitkumar1007 (via GitHub)" <gi...@apache.org>.
Sumitkumar1007 commented on issue #23622:
URL: https://github.com/apache/superset/issues/23622#issuecomment-1500939247
I am having the same issue with 2.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] masayyed commented on issue #23622: Failed run with 'superset db upgrade'
Posted by "masayyed (via GitHub)" <gi...@apache.org>.
masayyed commented on issue #23622:
URL: https://github.com/apache/superset/issues/23622#issuecomment-1504104498
I had same issue. After multiple tries, I got solution.
it was Sqlite version issue. Actually centos7 comes with very old version of Sqlite.
I have replaced Sqlite with postgres 15
`SQLALCHEMY_DATABASE_URI = 'postgresql://postgres:postgres@localhost/postgres' `
and it worked fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Failed run with 'superset db upgrade' [superset]
Posted by "StormingBullet (via GitHub)" <gi...@apache.org>.
StormingBullet closed issue #23622: Failed run with 'superset db upgrade'
URL: https://github.com/apache/superset/issues/23622
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] StormingBullet commented on issue #23622: Failed run with 'superset db upgrade'
Posted by "StormingBullet (via GitHub)" <gi...@apache.org>.
StormingBullet commented on issue #23622:
URL: https://github.com/apache/superset/issues/23622#issuecomment-1508127119
> Is it related to self signed certificate?
No. Just related to compiled openssh with missed root ca certificate chain. It was resolved by creating a link.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] leo-schick commented on issue #23622: Failed run with 'superset db upgrade'
Posted by "leo-schick (via GitHub)" <gi...@apache.org>.
leo-schick commented on issue #23622:
URL: https://github.com/apache/superset/issues/23622#issuecomment-1511561415
I have the same issue. This issue seems to be similar to #23483 . Looks like the superset db migration script needs fixing...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Failed run with 'superset db upgrade' [superset]
Posted by "jiortizv (via GitHub)" <gi...@apache.org>.
jiortizv commented on issue #23622:
URL: https://github.com/apache/superset/issues/23622#issuecomment-1745318303
Another option could be to edit the function "upgrade()" in "<python-install>/lib/pythonX.X/site-packages/superset/migrations/versions/2022-07-11_11-26_ffa79af61a56_rename_report_schedule_extra_to_extra_.py" from:
```
def upgrade():
op.alter_column(
"report_schedule",
"extra",
new_column_name="extra_json",
# existing info is required for MySQL
existing_type=Text,
existing_nullable=True,
)
```
to:
```
def upgrade():
with op.batch_alter_table('report_schedule') as bop:
bop.alter_column(
'extra',
new_column_name='extra_json',
# existing info is required for MySQL
existing_type=Text,
existing_nullable=True,
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Muyao-chen commented on issue #23622: Failed run with 'superset db upgrade'
Posted by "Muyao-chen (via GitHub)" <gi...@apache.org>.
Muyao-chen commented on issue #23622:
URL: https://github.com/apache/superset/issues/23622#issuecomment-1735225288
Comment out
/usr/local/lib/python3.8/site-packages/superset/migrations/versions/2022-07-11_11-26_ffa79af61a56_rename_report_schedule_extra_to_extra_.py"
of the two methods.
Manually create (need to delete first, if there is data, back up the data first) table report_schedule and modify the column (extra) to (extra_json)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org