You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/06/01 09:41:47 UTC
[GitHub] [superset] amilamanoj opened a new issue #14930: Cannot add a Dataset in Superset for a maprdb json table via Drill
amilamanoj opened a new issue #14930:
URL: https://github.com/apache/superset/issues/14930
Cannot add a Dataset in Superset for a maprdb json table via Drill.
### Expected results
A Dataset is created in Superset.
### Actual results
Dataset is not created. An error message is shown (see screenshot).
#### Screenshots
#### How to reproduce the bug
Preconditions:
- A database connection to a maprdb table via Drill is already created. "Test connection" is successful.
- Can successfully query the data in Superset SQL editor using `select * from dfs.tmp.sstest;`
1. Go to 'Data -> Dataset'.
2. Click on '+ Dataset'.
3. Configure Database, Schema and Table (as shown in the screenshot).
4. See error (See screenshot above, and stacktrace below).
### Environment
(please complete the following information):
- superset version: `1.1.0`
- python version: `3.7.9`
- node.js version: `Not found`
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
#### Data in example table:
select * from dfs.tmp.sstest;
```
+--------------+----------------------+------------------------+
| _id | studio | title |
+--------------+----------------------+------------------------+
| movie0000002 | Command Line Studios | Developers on the Edge |
+--------------+----------------------+------------------------+
```
#### Stacktrace:
```
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): host:8047
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'host'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
DEBUG:urllib3.connectionpool:https://host:8047 "POST /query.json HTTP/1.1" 200 219
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'host'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
DEBUG:urllib3.connectionpool:https://host:8047 "POST /query.json HTTP/1.1" 200 269
DEBUG:root:Mapping column SCHEMA_NAME of Drill type VARCHAR to dtype string
DEBUG:root:Mapping column TYPE of Drill type VARCHAR to dtype string
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'host'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
DEBUG:urllib3.connectionpool:https://host:8047 "POST /query.json HTTP/1.1" 200 192
DEBUG:root:Mapping column TABLE_NAME of Drill type VARCHAR to dtype string
ERROR:root:single positional indexer is out-of-bounds
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 84, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 80, in wraps
duration, response = time_function(f, self, *args, **kwargs)
File "/app/superset/utils/core.py", line 1368, in time_function
response = func(*args, **kwargs)
File "/app/superset/utils/log.py", line 224, in wrapper
value = f(*args, **kwargs)
File "/app/superset/datasets/api.py", line 236, in post
new_model = CreateDatasetCommand(g.user, item).run()
File "/app/superset/datasets/commands/create.py", line 47, in run
self.validate()
File "/app/superset/datasets/commands/create.py", line 88, in validate
database, table_name, schema
File "/app/superset/datasets/dao.py", line 81, in validate_table_exists
database.get_table(table_name, schema=schema)
File "/app/superset/models/core.py", line 608, in get_table
autoload_with=self.get_sqla_engine(),
File "<string>", line 2, in __new__
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/deprecations.py", line 139, in warned
return fn(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 560, in __new__
metadata._remove_table(name, schema)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 555, in __new__
table._init(name, metadata, *args, **kw)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 649, in _init
resolve_fks=resolve_fks,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 673, in _autoload
_extend_on=_extend_on,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2212, in run_callable
return conn.run_callable(callable_, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1653, in run_callable
return callable_(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 470, in reflecttable
table, include_columns, exclude_columns, resolve_fks, **opts
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/reflection.py", line 665, in reflecttable
table_name, schema, **table.dialect_kwargs
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/reflection.py", line 391, in get_columns
self.bind, table_name, schema, info_cache=self.info_cache, **kw
File "/usr/local/lib/python3.7/site-packages/sqlalchemy_drill/base.py", line 382, in get_columns
views = self.get_view_names(connection, schema)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy_drill/base.py", line 337, in get_view_names
curs = connection.execute("SELECT `TABLE_NAME` FROM INFORMATION_SCHEMA.views WHERE table_schema='" + schema + "'")
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1003, in execute
return self._execute_text(object_, multiparams, params)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1178, in _execute_text
parameters,
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1514, in _handle_dbapi_exception
util.raise_(exc_info[1], with_traceback=exc_info[2])
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy_drill/drilldbapi/_drilldbapi.py", line 65, in func_wrapper
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/sqlalchemy_drill/drilldbapi/_drilldbapi.py", line 165, in execute
elif str(df[col_name].iloc[0]).startswith("[") and str(df[col_name].iloc[0]).endswith("]"):
File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 895, in __getitem__
return self._getitem_axis(maybe_callable, axis=axis)
File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1501, in _getitem_axis
self._validate_integer(key, axis)
File "/usr/local/lib/python3.7/site-packages/pandas/core/indexing.py", line 1444, in _validate_integer
raise IndexError("single positional indexer is out-of-bounds")
IndexError: single positional indexer is out-of-bounds
10.130.2.1 - - [01/Jun/2021:09:14:17 +0000] "POST /api/v1/dataset/ HTTP/1.1" 500 26 "https://host/tablemodelv
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] amilamanoj edited a comment on issue #14930: Cannot add a Dataset for a maprdb json table via Drill
Posted by GitBox <gi...@apache.org>.
amilamanoj edited a comment on issue #14930:
URL: https://github.com/apache/superset/issues/14930#issuecomment-852959689
@junlincc @betodealmeida We can try come up with a fix, but we're not yet familiar with Superset internals. Can you see where could be the issue looking at the stacktrace?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] amilamanoj commented on issue #14930: Cannot add a Dataset for a maprdb json table via Drill
Posted by GitBox <gi...@apache.org>.
amilamanoj commented on issue #14930:
URL: https://github.com/apache/superset/issues/14930#issuecomment-852959689
@junlincc We can try come up with a fix, but we're not yet familiar with Superset internals. Can you see where could be the issue looking at the stacktrace?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] amilamanoj closed issue #14930: Cannot add a Dataset for a maprdb json table via Drill
Posted by GitBox <gi...@apache.org>.
amilamanoj closed issue #14930:
URL: https://github.com/apache/superset/issues/14930
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] amilamanoj edited a comment on issue #14930: Cannot add a Dataset for a maprdb json table via Drill
Posted by GitBox <gi...@apache.org>.
amilamanoj edited a comment on issue #14930:
URL: https://github.com/apache/superset/issues/14930#issuecomment-852959689
@junlincc @betodealmeida We can try come up with a fix, but we're not yet familiar with Superset internals. Are you able to provide a hint where could be the issue (hopefully stacktrace helps)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] junlincc edited a comment on issue #14930: Cannot add a Dataset for a maprdb json table via Drill
Posted by GitBox <gi...@apache.org>.
junlincc edited a comment on issue #14930:
URL: https://github.com/apache/superset/issues/14930#issuecomment-852470084
@amilamanoj please feel free to open a PR for this issue, as we may not be able to work on it soon. 🙏 cc @betodealmeida
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] junlincc commented on issue #14930: Cannot add a Dataset for a maprdb json table via Drill
Posted by GitBox <gi...@apache.org>.
junlincc commented on issue #14930:
URL: https://github.com/apache/superset/issues/14930#issuecomment-852470084
@amilamanoj please feel free to open a PR for this issue, as we may not be able to work on it soon. 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] amilamanoj commented on issue #14930: Cannot add a Dataset for a maprdb json table via Drill
Posted by GitBox <gi...@apache.org>.
amilamanoj commented on issue #14930:
URL: https://github.com/apache/superset/issues/14930#issuecomment-860237235
https://github.com/JohnOmernik/sqlalchemy-drill/pull/62 resolves this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] srinify commented on issue #14930: Cannot add a Dataset for a maprdb json table via Drill
Posted by GitBox <gi...@apache.org>.
srinify commented on issue #14930:
URL: https://github.com/apache/superset/issues/14930#issuecomment-855429001
hey @cgivre any advice here? 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org