You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/04/02 05:36:41 UTC
[GitHub] [superset] nightbear1009 opened a new issue #19496: migration plan " New dataset models " can't execute properly
nightbear1009 opened a new issue #19496:
URL: https://github.com/apache/superset/issues/19496
A clear and concise description of what the bug is.
#### How to reproduce the bug
1. add new dataset using virtual query with bigquery
2. the sql is as below
```
SELECT
project_id,
user_email,
creation_time,
total_bytes_billed,
query
FROM `region-us`.INFORMATION_SCHEMA.JOBS_BY_PROJECT
WHERE job_type = "QUERY"
```
3. go to console call `superset db upgrade`
### Expected results
success
### Actual results
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/pybigquery/sqlalchemy_bigquery.py", line 839, in _get_table
table = client.get_table(table_ref)
File "/usr/local/lib/python3.8/site-packages/google/cloud/bigquery/client.py", line 1008, in get_table
api_response = self._call_api(
File "/usr/local/lib/python3.8/site-packages/google/cloud/bigquery/client.py", line 756, in _call_api
return call()
File "/usr/local/lib/python3.8/site-packages/google/api_core/retry.py", line 283, in retry_wrapped_func
return retry_target(
File "/usr/local/lib/python3.8/site-packages/google/api_core/retry.py", line 190, in retry_target
return target()
File "/usr/local/lib/python3.8/site-packages/google/cloud/_http/__init__.py", line 480, in api_request
raise exceptions.from_http_response(response)
google.api_core.exceptions.NotFound: 404 GET https://bigquery.googleapis.com/bigquery/v2/projects/xxx_projectId/datasets/INFORMATION_SCHEMA/tables/JOBS_BY_PROJECT?prettyPrint=false: Not found: Table xxx_projectId:INFORMATION_SCHEMA.JOBS_BY_PROJECT
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/superset", line 33, in <module>
sys.exit(load_entry_point('apache-superset', 'console_scripts', 'superset')())
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 601, in main
return super().main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 445, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_migrate/cli.py", line 149, in upgrade
_upgrade(directory, revision, sql, tag, x_arg)
File "/usr/local/lib/python3.8/site-packages/flask_migrate/__init__.py", line 98, in wrapped
f(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_migrate/__init__.py", line 185, in upgrade
command.upgrade(config, revision, sql=sql, tag=tag)
File "/usr/local/lib/python3.8/site-packages/alembic/command.py", line 294, in upgrade
script.run_env()
File "/usr/local/lib/python3.8/site-packages/alembic/script/base.py", line 490, in run_env
util.load_python_file(self.dir, "env.py")
File "/usr/local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file
module = load_module_py(module_id, path)
File "/usr/local/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 843, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/app/superset/extensions/../migrations/env.py", line 126, in <module>
run_migrations_online()
File "/app/superset/extensions/../migrations/env.py", line 118, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/usr/local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 813, in run_migrations
self.get_context().run_migrations(**kw)
File "/usr/local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 561, in run_migrations
step.migration_fn(**kw)
File "/app/superset/migrations/versions/b8d3a24d9131_new_dataset_models.py", line 624, in upgrade
after_insert(target=dataset)
File "/app/superset/migrations/versions/b8d3a24d9131_new_dataset_models.py", line 422, in after_insert
tables = load_or_create_tables(
File "/app/superset/migrations/versions/b8d3a24d9131_new_dataset_models.py", line 282, in load_or_create_tables
column_metadata = inspector.get_columns(table.table, schema=table.schema)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/reflection.py", line 390, in get_columns
col_defs = self.dialect.get_columns(
File "/usr/local/lib/python3.8/site-packages/pybigquery/sqlalchemy_bigquery.py", line 872, in get_columns
table = self._get_table(connection, table_name, schema)
File "/usr/local/lib/python3.8/site-packages/pybigquery/sqlalchemy_bigquery.py", line 841, in _get_table
raise NoSuchTableError(table_name)
sqlalchemy.exc.NoSuchTableError: JOBS_BY_PROJECT
#### Screenshots
If applicable, add screenshots to help explain your problem.
### Environment
(please complete the following information):
current superset version is helm install lastest version, I think is 1.4.2-rc1
- browser type and version:
- superset version: `superset version`
- python version: `python --version`
- node.js version: `node -v`
- any feature flags active:
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [ y] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [ y] I have reproduced the issue with at least the latest released version of superset.
- [ y] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org