You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/10/11 18:46:35 UTC
[GitHub] [superset] FromTheCosmos opened a new issue #17059: Superset 0.39 => 1.3.0 upgrade
FromTheCosmos opened a new issue #17059:
URL: https://github.com/apache/superset/issues/17059
When upgrading from 0.39 superset on Ubuntu 18.04 and upgrading to Superset 1.3.0 on Ubuntu 20.04, there is a problem with data being moved from one superset instance to another.
#### How to reproduce the bug
1. Copy `superset.db` from Ubuntu 18.04 system
2. Move a copy of `superset.db` to the Ubuntu 20.04 instance with superset 1.3.0 in the `etc/superset/.superset/superset.db` location
3. Activate the virtual environment and run `superset db upgrade` and `superset init`
4. The `superset db upgrade` finishes without error, however the `superset init` command does not complete due to decryption key error.
```
$ . /etc/superset/bin/activate
(superset) $ superset db upgrade
logging was configured successfully
2021-10-11 18:34:46,763:INFO:superset.utils.logging_configurator:logging was configured successfully
2021-10-11 18:34:46,829:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/etc/superset/lib/python3.8/site-packages/flask_caching/__init__.py:201: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
warnings.warn(
No PIL installation found
2021-10-11 18:34:48,917:INFO:superset.utils.screenshots:No PIL installation found
WARNI [alembic.env] SQLite Database support for metadata databases will be removed in a future version of Superset.
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
(superset) $ superset init
logging was configured successfully
2021-10-11 18:35:56,758:INFO:superset.utils.logging_configurator:logging was configured successfully
2021-10-11 18:35:57,082:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/etc/superset/lib/python3.8/site-packages/flask_caching/__init__.py:201: UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively disabled.
warnings.warn(
No PIL installation found
2021-10-11 18:35:58,187:INFO:superset.utils.screenshots:No PIL installation found
Syncing role definition
2021-10-11 18:36:00,085:INFO:superset.security.manager:Syncing role definition
Syncing Admin perms
2021-10-11 18:36:00,096:INFO:superset.security.manager:Syncing Admin perms
Syncing Alpha perms
2021-10-11 18:36:00,290:INFO:superset.security.manager:Syncing Alpha perms
Syncing Gamma perms
2021-10-11 18:36:00,372:INFO:superset.security.manager:Syncing Gamma perms
Syncing granter perms
2021-10-11 18:36:00,453:INFO:superset.security.manager:Syncing granter perms
Syncing sql_lab perms
2021-10-11 18:36:00,534:INFO:superset.security.manager:Syncing sql_lab perms
Fetching a set of all perms to lookup which ones are missing
2021-10-11 18:36:00,612:INFO:superset.security.manager:Fetching a set of all perms to lookup which ones are missing
Creating missing datasource permissions.
2021-10-11 18:36:00,686:INFO:superset.security.manager:Creating missing datasource permissions.
Traceback (most recent call last):
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe6 in position 1: invalid continuation byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/etc/superset/bin/superset", line 8, in <module>
sys.exit(superset())
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/flask/cli.py", line 426, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/flask/cli.py", line 426, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/superset/cli.py", line 90, in init
security_manager.sync_role_definitions()
File "/etc/superset/lib/python3.8/site-packages/superset/security/manager.py", line 704, in sync_role_definitions
self.create_missing_perms()
File "/etc/superset/lib/python3.8/site-packages/superset/security/manager.py", line 648, in create_missing_perms
merge_pv("datasource_access", datasource.get_perm())
File "/etc/superset/lib/python3.8/site-packages/superset/connectors/sqla/models.py", line 621, in get_perm
return f"[{self.database}].[{self.table_name}](id:{self.id})"
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 294, in __get__
return self.impl.get(instance_state(instance), dict_)
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 730, in get
value = self.callable_(state, passive)
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line 759, in _load_for_state
return self._emit_lazyload(
File "<string>", line 1, in <lambda>
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line 847, in _emit_lazyload
q(session)
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/ext/baked.py", line 615, in _load_on_pk_identity
result = list(bq.for_session(self.session).params(**params))
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
cursor.close()
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
rows = [proc(row) for row in fetch]
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
rows = [proc(row) for row in fetch]
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 579, in _instance
_populate_full(
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 725, in _populate_full
dict_[key] = getter(row)
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1278, in process
return process_value(impl_processor(value), dialect)
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 469, in process_result_value
value = super().process_result_value(value=value, dialect=dialect)
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 414, in process_result_value
decrypted_value = self.engine.decrypt(value)
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key
```
### Expected results
I expect superset to be able to upgrade and use `superset.db` for the init without it prompting error.
### Actual results
Invalid decryption key error appears.
### Environment
- superset version: `1.3.0`
- python version: `Python 3.8.10`
- ubuntu 20.04
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar. (There are similar ones, but no solution)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] FromTheCosmos commented on issue #17059: Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key
Posted by GitBox <gi...@apache.org>.
FromTheCosmos commented on issue #17059:
URL: https://github.com/apache/superset/issues/17059#issuecomment-941286389
> My guess is the `SECRET_KEY` got changed. If you have the value from your 0.39 instance, you should try using setting that same value.
>
> Here is some additional reading:
>
> * [[Question] What is SECRET_KEY used in superset for? #7190](https://github.com/apache/superset/issues/7190)
>
> * [Change SECRET_KEY and get error `ValueError: Invalid decryption key` on K8s #8538](https://github.com/apache/superset/issues/8538)
I read through those issues but they were specific to postgres. I am using SQLite. Additionally, they do not really explain where/how to update the key. `superset_config.py` is the same on both instances, so there shouldn't be configuration differences here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] FromTheCosmos commented on issue #17059: Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key
Posted by GitBox <gi...@apache.org>.
FromTheCosmos commented on issue #17059:
URL: https://github.com/apache/superset/issues/17059#issuecomment-941286389
> My guess is the `SECRET_KEY` got changed. If you have the value from your 0.39 instance, you should try using setting that same value.
>
> Here is some additional reading:
>
> * [[Question] What is SECRET_KEY used in superset for? #7190](https://github.com/apache/superset/issues/7190)
>
> * [Change SECRET_KEY and get error `ValueError: Invalid decryption key` on K8s #8538](https://github.com/apache/superset/issues/8538)
I read through those issues but they were specific to postgres. I am using SQLite. Additionally, they do not really explain where/how to update the key. `superset_config.py` is the same on both instances, so there shouldn't be configuration differences here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] junlincc closed issue #17059: Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key
Posted by GitBox <gi...@apache.org>.
junlincc closed issue #17059:
URL: https://github.com/apache/superset/issues/17059
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] jhult commented on issue #17059: Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key
Posted by GitBox <gi...@apache.org>.
jhult commented on issue #17059:
URL: https://github.com/apache/superset/issues/17059#issuecomment-941274601
My guess is the `SECRET_KEY` got changed. If you have the value from your 0.39 instance, you should try using setting that same value.
Here is some additional reading:
- #7190
- #8538
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] jhult commented on issue #17059: Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key
Posted by GitBox <gi...@apache.org>.
jhult commented on issue #17059:
URL: https://github.com/apache/superset/issues/17059#issuecomment-941274601
My guess is the `SECRET_KEY` got changed. If you have the value from your 0.39 instance, you should try using setting that same value.
Here is some additional reading:
- #7190
- #8538
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] junlincc commented on issue #17059: Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key
Posted by GitBox <gi...@apache.org>.
junlincc commented on issue #17059:
URL: https://github.com/apache/superset/issues/17059#issuecomment-941281814
@jhult Thank you for helping out with answer. 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] junlincc closed issue #17059: Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key
Posted by GitBox <gi...@apache.org>.
junlincc closed issue #17059:
URL: https://github.com/apache/superset/issues/17059
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] junlincc commented on issue #17059: Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key
Posted by GitBox <gi...@apache.org>.
junlincc commented on issue #17059:
URL: https://github.com/apache/superset/issues/17059#issuecomment-941281814
@jhult Thank you for helping out with answer. 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org