You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "Heyzi (via GitHub)" <gi...@apache.org> on 2023/09/25 14:05:14 UTC
[GitHub] [superset] Heyzi opened a new issue, #25394: Superset 2.1.0 -> 3.0.0 migration error
Heyzi opened a new issue, #25394:
URL: https://github.com/apache/superset/issues/25394
after upgrading to version 3.0.0 we get errors when updating the database
#### How to reproduce the bug
1. changed the superset image to version 3.0.0
2. docker-compose up
### Expected results
no error
### Actual results
`
NFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade f3c2d8ec8595 -> 9c2a5681ddfd, convert key-value entries to json
Loaded your LOCAL configuration at [/superset/docker/configs/superset_config.py]
Traceback (most recent call last):
File "/superset/app/venv/bin/superset", line 8, in <module>
sys.exit(superset())
File "/superset/app/venv/lib64/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/superset/app/venv/lib64/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/superset/app/venv/lib64/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/superset/app/venv/lib64/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/superset/app/venv/lib64/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/superset/app/venv/lib64/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/superset/app/venv/lib64/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/superset/app/venv/lib64/python3.9/site-packages/flask/cli.py", line 357, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/superset/app/venv/lib64/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/superset/app/venv/lib64/python3.9/site-packages/flask_migrate/cli.py", line 149, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "/superset/app/venv/lib64/python3.9/site-packages/flask_migrate/__init__.py", line 98, in wrapped
f(*args, **kwargs)
File "/superset/app/venv/lib64/python3.9/site-packages/flask_migrate/__init__.py", line 185, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/superset/app/venv/lib64/python3.9/site-packages/alembic/command.py", line 294, in upgrade
script.run_env()
File "/superset/app/venv/lib64/python3.9/site-packages/alembic/script/base.py", line 490, in run_env
util.load_python_file(self.dir, "env.py")
File "/superset/app/venv/lib64/python3.9/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
module = load_module_py(module_id, path)
File "/superset/app/venv/lib64/python3.9/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 850, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/superset/app/venv/lib64/python3.9/site-packages/superset/extensions/../migrations/env.py", line 126, in <module>
run_migrations_online()
File "/superset/app/venv/lib64/python3.9/site-packages/superset/extensions/../migrations/env.py", line 118, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/superset/app/venv/lib64/python3.9/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
self.get_context().run_migrations(**kw)
File "/superset/app/venv/lib64/python3.9/site-packages/alembic/runtime/migration.py", line 561, in run_migrations
step.migration_fn(**kw)
File "/superset/app/venv/lib64/python3.9/site-packages/superset/migrations/versions/2023-05-01_12-03_9c2a5681ddfd_convert_key_value_entries_to_json.py", line 71, in upgrade
value = RestrictedUnpickler(io.BytesIO(entry.value)).load() or {}
File "/superset/app/venv/lib64/python3.9/site-packages/superset/migrations/versions/2023-05-01_12-03_9c2a5681ddfd_convert_key_value_entries_to_json.py", line 49, in find_class
raise pickle.UnpicklingError(f"Unpickling of {module}.{name} is forbidden")
_pickle.UnpicklingError: Unpickling of builtins.getattr is forbidden
`
### Environment
- browser type and version: Google chrome 117
- superset version: 3.0.0
- python version: 3.9
- any feature flags active:
"DASHBOARD_RBAC": True,
"ESCAPE_MARKDOWN_HTML": False,
"ENABLE_TEMPLATE_PROCESSING": True,
"LISTVIEWS_DEFAULT_CARD_VIEW": True,
"SCHEDULED_QUERIES": True,
"SQL_VALIDATORS_BY_ENGINE": False,
"SQLLAB_BACKEND_PERSISTENCE": True,
"THUMBNAILS": False,
"THUMBNAILS_SQLA_LISTENERS": False,
"HORIZONTAL_FILTER_BAR": True,
"DASHBOARD_VIRTUALIZATION": True,
"DRILL_BY": True,
"ALLOW_FULL_CSV_EXPORT": True,
"CACHE_IMPERSONATION": True,
"CONFIRM_DASHBOARD_DIFF": True,
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
Everythong work fine on 2.1.0 version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Superset 2.1.0 -> 3.0.0 migration error [superset]
Posted by "nathan-gilbert (via GitHub)" <gi...@apache.org>.
nathan-gilbert commented on issue #25394:
URL: https://github.com/apache/superset/issues/25394#issuecomment-1836799558
Anyone ever find an answer for this? I'm still seeing this error on 3.0.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Superset 2.1.0 -> 3.0.0 migration error [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #25394:
URL: https://github.com/apache/superset/issues/25394#issuecomment-2065315672
Sounds like you got past the problem, and also we now support Superset 3.1 and 4.0. Let us know if you have any trouble with those versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] nathan-gilbert commented on issue #25394: Superset 2.1.0 -> 3.0.0 migration error
Posted by "nathan-gilbert (via GitHub)" <gi...@apache.org>.
nathan-gilbert commented on issue #25394:
URL: https://github.com/apache/superset/issues/25394#issuecomment-1734555980
I have the same issue on a PYPI install and attempting to upgrade from 2.1.0 to 3.0.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Heyzi commented on issue #25394: Superset 2.1.0 -> 3.0.0 migration error
Posted by "Heyzi (via GitHub)" <gi...@apache.org>.
Heyzi commented on issue #25394:
URL: https://github.com/apache/superset/issues/25394#issuecomment-1738535586
We have about 10 separate apache superset instances, but with different user groups.
Only one installation had a problem with migrations, I don't know what users could have broken there :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Superset 2.1.0 -> 3.0.0 migration error [superset]
Posted by "nathan-gilbert (via GitHub)" <gi...@apache.org>.
nathan-gilbert commented on issue #25394:
URL: https://github.com/apache/superset/issues/25394#issuecomment-1836857142
I got passed the error by modifying: `2023-05-01_12-03_9c2a5681ddfd_convert_key_value_entries_to_json.py`
I added lines 78-80
```python
except pickle.UnpicklingError as ex:
73 if str(ex) == "pickle data was truncated":
74 # make truncated values that were created prior to #20385 an empty
75 # dict so that downgrading will work properly.
76 truncated_count += 1
77 value = {}
78 elif str(ex) == "Unpickling of builtins.getattr is forbidden":
79 truncated_count += 1
80 value = {}
81 else:
82 raise
```
After running the upgrades all my data appeared to be preserved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Superset 2.1.0 -> 3.0.0 migration error [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #25394: Superset 2.1.0 -> 3.0.0 migration error
URL: https://github.com/apache/superset/issues/25394
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org