You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "Subramanian-R-Needl (via GitHub)" <gi...@apache.org> on 2024/01/24 06:27:43 UTC
[I] tables.normalize_columns does not exist [superset]
Subramanian-R-Needl opened a new issue, #26774:
URL: https://github.com/apache/superset/issues/26774
### Bug description
I'm using `apache/superset:latest-dev` image as base image and made some changes to `config.py` file to enable Google OAuth for Superset. After enabling it, I'm able to login and use SQL Lab. But when I try to open any dashboard or chart, I'm facing the error
```
There was an issue fetching your recent activity:
There was an issue fetching your chart: [object Response]
```
When I checked the logs, found this error
```[SQL: SELECT tables.uuid AS tables_uuid, tables.created_on AS tables_created_on, tables.changed_on AS tables_changed_on, tables.id AS tables_id, tables.description AS tables_description, tables.default_endpoint AS tables_default_endpoint, tables.is_featured AS tables_is_featured, tables.filter_select_enabled AS tables_filter_select_enabled, tables."offset" AS tables_offset, tables.cache_timeout AS tables_cache_timeout, tables.params AS tables_params, tables.perm AS tables_perm, tables.schema_perm AS tables_schema_perm, tables.is_managed_externally AS tables_is_managed_externally, tables.external_url AS tables_external_url, tables.table_name AS tables_table_name, tables.main_dttm_col AS tables_main_dttm_col, tables.database_id AS tables_database_id, tables.fetch_values_predicate AS tables_fetch_values_predicate, tables.schema AS tables_schema, tables.sql AS tables_sql, tables.is_sqllab_view AS tables_is_sqllab_view, tables.template_params AS tables_template_params, tables.extra
AS tables_extra, tables.normalize_columns AS tables_normalize_columns, tables.always_filter_main_dttm AS tables_always_filter_main_dttm, tables.created_by_fk AS tables_created_by_fk, tables.changed_by_fk AS tables_changed_by_fk, anon_1.slices_datasource_id AS anon_1_slices_datasource_id, anon_1.slices_datasource_type AS anon_1_slices_datasource_type
FROM (SELECT DISTINCT slices.datasource_id AS slices_datasource_id, slices.datasource_type AS slices_datasource_type
FROM slices, dashboard_slices
WHERE %(param_1)s = dashboard_slices.dashboard_id AND slices.id = dashboard_slices.slice_id) AS anon_1 JOIN tables ON anon_1.slices_datasource_id = tables.id AND anon_1.slices_datasource_type = %(datasource_type_1)s]
[parameters: {'param_1': 7, 'datasource_type_1': 'table'}]
(Background on this error at: https://sqlalche.me/e/14/f405)
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column tables.normalize_columns does not exist
LINE 1: ...es_template_params, tables.extra AS tables_extra, tables.nor...
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/superset/views/base.py", line 251, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 487, in get_list_headless
duration, response = time_function(super().get_list_headless, **kwargs)
File "/app/superset/utils/core.py", line 1454, in time_function
response = func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/flask_appbuilder/api/__init__.py", line 1613, in get_list_headless
response[API_RESULT_RES_KEY] = list_model_schema.dump(lst, many=True)
File "/usr/local/lib/python3.9/site-packages/marshmallow/schema.py", line 557, in dump
result = self._serialize(processed_obj, many=many)
File "/usr/local/lib/python3.9/site-packages/marshmallow/schema.py", line 519, in _serialize
return [
File "/usr/local/lib/python3.9/site-packages/marshmallow/schema.py", line 520, in <listcomp>
self._serialize(d, many=False)
File "/usr/local/lib/python3.9/site-packages/marshmallow/schema.py", line 525, in _serialize
value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
File "/usr/local/lib/python3.9/site-packages/marshmallow/fields.py", line 336, in serialize
value = self.get_value(obj, attr, accessor=accessor)
File "/usr/local/lib/python3.9/site-packages/marshmallow/fields.py", line 265, in get_value
return accessor_func(obj, check_key, default)
File "/usr/local/lib/python3.9/site-packages/marshmallow/schema.py", line 484, in get_attribute
return get_value(obj, attr, default)
File "/usr/local/lib/python3.9/site-packages/marshmallow/utils.py", line 271, in get_value
return _get_value_for_key(obj, key, default)
File "/usr/local/lib/python3.9/site-packages/marshmallow/utils.py", line 285, in _get_value_for_key
return getattr(obj, key, default)
File "/app/superset/models/dashboard.py", line 245, in thumbnail_url
return f"/api/v1/dashboard/{self.id}/thumbnail/{self.digest}/"
File "/app/superset/models/dashboard.py", line 237, in digest
return get_dashboard_digest(self)
File "/app/superset/thumbnails/digest.py", line 63, in get_dashboard_digest
f"{dashboard.id}\n{dashboard.charts}\n{dashboard.position_json}\n"
File "/app/superset/models/dashboard.py", line 215, in charts
return [slc.chart for slc in self.slices]
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/attributes.py", line 481, in __get__
return self.impl.get(state, dict_)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/attributes.py", line 941, in get
value = self._fire_loader_callables(state, key, passive)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/attributes.py", line 977, in _fire_loader_callables
return self.callable_(state, passive)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/strategies.py", line 911, in _load_for_state
return self._emit_lazyload(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/strategies.py", line 1051, in _emit_lazyload
result = result.unique().scalars().all()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 1371, in all
return self._allrows()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 401, in _allrows
rows = self._fetchall_impl()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 1284, in _fetchall_impl
return self._real_result._fetchall_impl()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 1696, in _fetchall_impl
return list(self.iterator)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/loading.py", line 151, in chunks
rows = [proc(row) for row in fetch]
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/loading.py", line 151, in <listcomp>
rows = [proc(row) for row in fetch]
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/loading.py", line 962, in _instance
_populate_full(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/loading.py", line 1136, in _populate_full
populator(state, dict_, row)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/strategies.py", line 1894, in load_scalar_from_subq
collection = collections.get(tuple_getter(row), (None,))
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/strategies.py", line 1601, in get
self._load()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/strategies.py", line 1617, in _load
rows = list(q.params(self.params))
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2896, in __iter__
return self._iter().__iter__()
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/query.py", line 2903, in _iter
result = self.session.execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1696, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
return connection._execute_clauseelement(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2043, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column tables.normalize_columns does not exist
LINE 1: ...es_template_params, tables.extra AS tables_extra, tables.nor...
^ ^
```
I tried running `db upgrade` in the Superset pod, but that also failed
```
kubectl exec -it superset-service-7f64769bb-g29k2 -- superset db upgrade
logging was configured successfully
2024-01-24 06:23:39,259:INFO:superset.utils.logging_configurator:logging was configured successfully
2024-01-24 06:23:39,584:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/usr/local/lib/python3.9/site-packages/flask_limiter/extension.py:293: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
warnings.warn(
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
ERROR [flask_migrate] Error: Can't locate revision identified by '10c30ec09073'
command terminated with exit code 1
```
### How to reproduce the bug
1. Create a dockerfile with `apache/superset:latest-dev` as base image
2. Enable OAuth Authentication method in `config.py`
3. Build and Push it to your Container registry and update the superset k8s deployment in your k8s cluster
4. Login to Superset
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
_No response_
### Checklist
- [ ] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] tables.normalize_columns does not exist [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina closed issue #26774: tables.normalize_columns does not exist
URL: https://github.com/apache/superset/issues/26774
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] tables.normalize_columns does not exist [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on issue #26774:
URL: https://github.com/apache/superset/issues/26774#issuecomment-2142860800
@shenrie When you say "a fresh install of Superset 4.0.1" do you mean an empty metadata database (running all migrations from scratch) or are you upgrading an existing metadata database by executing the migrations contained in 4.0.1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] tables.normalize_columns does not exist [superset]
Posted by "shenrie (via GitHub)" <gi...@apache.org>.
shenrie commented on issue #26774:
URL: https://github.com/apache/superset/issues/26774#issuecomment-2142779481
@michael-s-molina I am experiencing this same error with a fresh install of Superset 4.0.1. Any thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] tables.normalize_columns does not exist [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on issue #26774:
URL: https://github.com/apache/superset/issues/26774#issuecomment-1947423706
Hi @Subramanian-R-Needl. The original problem is caused by the fact the you were unable to run `superset db upgrade` and [this](https://github.com/apache/superset/blob/acb2d1d6b4f26a9dfe2957970889af3950ec4084/superset/migrations/versions/2023-08-14_09-38_9f4a086c2676_add_normalize_columns_to_sqla_model.py#L19) migration was not run.
The reason you were not able to run the upgrade is because you're referencing a unexistent revision identifier ([10c30ec09073](https://github.com/search?q=repo%3Aapache%2Fsuperset+path%3A%2F%5Esuperset%5C%2Fmigrations%5C%2Fversions%5C%2F%2F+10c30ec09073&type=code)) in either `alembic_version` table or by manually modifying a migration file. If it's the table, you should update the revision identifier to the last migration revision you were able to run and execute `superset db upgrade` again.
I'm closing the issue as it's something specific to your deployment but feel free to reply here if that's not the case. Hope you are able to fix the problem with the provided instructions 🤞🏼.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org