You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/12/02 12:50:17 UTC
[GitHub] [superset] davidkhala opened a new issue, #22316: Apache Hive in OCI BDS
davidkhala opened a new issue, #22316:
URL: https://github.com/apache/superset/issues/22316
I have setup an Oracle Cloud Big Data Service to privide a Hive. I want to connect it.
The test connection looks good, but before I finish the database connection, it prompts
#### How to reproduce the bug
1. Go to 'Connect a database'
2. Choose 'Apache Hive' from 'SUPPORTED DATABASES' dropdown
3. Input the SQLALCHEMY URI as `hive://hive@168.138.166.53:10000`
4. Click "Test Connection", it will success
5. Click "Connect"
6. See error
### Expected results
Silent success
### Actual results
An error popup
`An error occurred while fetching databases: Object of type bytes is not JSON serializable`
#### Screenshots
### Environment
(please complete the following information):
- browser type and version: MS Edge
- superset version: master
- python version: docker image inline
- node.js version: docker image inline
- any feature flags active:
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
python stacktraces
```
superset_app | 2022-12-02 12:45:56,017:INFO:pyhive.hive:SHOW SCHEMAS
superset_app | 2022-12-02 12:45:56,017:DEBUG:pyhive.hive:TExecuteStatementReq(sessionHandle=TSessionHandle(sessionId=THandleIdentifier(guid=b'\xff\xeeQ\xdc0\xe2F\xc7\xbe\xd7hU\xad\x103\xb1', secret=b'\xdeaVP\xc6\x07G\xc3\x823DOG\x9b\xc9\x02')), statement='SHOW SCHEMAS', confOverlay=None, runAsync=False, queryTimeout=0)
superset_app | 2022-12-02 12:45:56,110:DEBUG:pyhive.hive:TExecuteStatementResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), operationHandle=TOperationHandle(operationId=THandleIdentifier(guid=b'\xefx\\\x88\xa1\xd8@m\x9a=\x89\x1f\xd6I\x95\x81', secret=b'-K\x01\x93G\x11N\x91\x8d\xc3\xce\x87\xd9\x19\x14\xfc'), operationType=0, hasResultSet=True, modifiedRowCount=None))
superset_app | 2022-12-02 12:45:56,111:DEBUG:pyhive.hive:TGetResultSetMetadataResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), schema=TTableSchema(columns=[TColumnDesc(columnName='database_name', typeDesc=TTypeDesc(types=[TTypeEntry(primitiveEntry=TPrimitiveTypeEntry(type=7, typeQualifiers=None), arrayEntry=None, mapEntry=None, structEntry=None, unionEntry=None, userDefinedTypeEntry=None)]), position=1, comment='from deserializer')]))
superset_app | 2022-12-02 12:45:56,113:DEBUG:pyhive.hive:TFetchResultsResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), hasMoreRows=False, results=TRowSet(startRowOffset=0, rows=[], columns=[TColumn(boolVal=None, byteVal=None, i16Val=None, i32Val=None, i64Val=None, doubleVal=None, stringVal=TStringColumn(values=['ctm', 'default', 'information_schema', 'sys'], nulls=b'\x00'), binaryVal=None)], binaryColumns=None, columnCount=None))
superset_app | 2022-12-02 12:45:56,113:DEBUG:pyhive.hive:TFetchResultsResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None), hasMoreRows=False, results=TRowSet(startRowOffset=0, rows=[], columns=[TColumn(boolVal=None, byteVal=None, i16Val=None, i32Val=None, i64Val=None, doubleVal=None, stringVal=TStringColumn(values=[], nulls=b'\x00'), binaryVal=None)], binaryColumns=None, columnCount=None))
superset_app | 2022-12-02 12:45:56,123:DEBUG:pyhive.hive:TCloseOperationResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None))
superset_app | 2022-12-02 12:45:56,126:DEBUG:pyhive.hive:TCloseSessionResp(status=TStatus(statusCode=0, infoMessages=None, sqlState=None, errorCode=None, errorMessage=None))
superset_app | 2022-12-02 12:45:56,165:ERROR:root:Object of type bytes is not JSON serializable
superset_app | Traceback (most recent call last):
superset_app | File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/__init__.py", line 86, in wraps
superset_app | return f(self, *args, **kwargs)
superset_app | File "/app/superset/views/base_api.py", line 114, in wraps
superset_app | raise ex
superset_app | File "/app/superset/views/base_api.py", line 111, in wraps
superset_app | duration, response = time_function(f, self, *args, **kwargs)
superset_app | File "/app/superset/utils/core.py", line 1604, in time_function
superset_app | response = func(*args, **kwargs)
superset_app | File "/app/superset/utils/log.py", line 265, in wrapper
superset_app | value = f(*args, **kwargs)
superset_app | File "/app/superset/views/base_api.py", line 84, in wraps
superset_app | return f(self, *args, **kwargs)
superset_app | File "/app/superset/databases/api.py", line 281, in post
superset_app | return self.response(201, id=new_model.id, result=item)
superset_app | File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/api/__init__.py", line 704, in response
superset_app | _ret_json = jsonify(kwargs)
superset_app | File "/usr/local/lib/python3.8/site-packages/flask/json/__init__.py", line 361, in jsonify
superset_app | f"{dumps(data, indent=indent, separators=separators)}\n",
superset_app | File "/usr/local/lib/python3.8/site-packages/flask/json/__init__.py", line 139, in dumps
superset_app | rv = _json.dumps(obj, **kwargs)
superset_app | File "/usr/local/lib/python3.8/json/__init__.py", line 234, in dumps
superset_app | return cls(
superset_app | File "/usr/local/lib/python3.8/json/encoder.py", line 199, in encode
superset_app | chunks = self.iterencode(o, _one_shot=True)
superset_app | File "/usr/local/lib/python3.8/json/encoder.py", line 257, in iterencode
superset_app | return _iterencode(o, 0)
superset_app | File "/usr/local/lib/python3.8/site-packages/flask/json/__init__.py", line 57, in default
superset_app | return super().default(o)
superset_app | File "/usr/local/lib/python3.8/json/encoder.py", line 179, in default
superset_app | raise TypeError(f'Object of type {o.__class__.__name__} '
superset_app | TypeError: Object of type bytes is not JSON serializable
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Connect to Apache Hive endpoint in OCI BDS [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1986252360
It sounds like this is likely fixed by now, and is pretty out of date if not. If people are still encountering this in current versions (3.x) please open a new Issue with updated context or a PR to address the problem. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] kashifjavedaddo commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "kashifjavedaddo (via GitHub)" <gi...@apache.org>.
kashifjavedaddo commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1506479295
Hi everyone,
We have also started facing this issue after upgrading to apache superset 2.1.0
Is there any resolution or workaround to connect to Apache Hive from superset?
We will appreciate an update as there is no reply on this thread regarding possible solution or workaround for almost 4 months now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Connect to Apache Hive endpoint in OCI BDS [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #22316: Connect to Apache Hive endpoint in OCI BDS
URL: https://github.com/apache/superset/issues/22316
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] kashifjavedaddo commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "kashifjavedaddo (via GitHub)" <gi...@apache.org>.
kashifjavedaddo commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1507943782
Hi everyone,
We have also started facing this issue after upgrading to apache superset 2.1.0
Is there any resolution or workaround to connect to Apache Hive from superset? The connection was working fine before upgrade in version 2.0.1
We will appreciate an update as there is no reply on the github thread regarding possible solution or workaround for almost 4 months now.
Regards,
Kashif Javed Rana
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] davidkhala commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by GitBox <gi...@apache.org>.
davidkhala commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1336313038
The same error also applies to homebrew Hive deployment.
I also checked it happened even with
```
cd superset
git checkout 1.4.0
TAG=1.4.0 docker compose -f docker-compose-non-dev.yml up
```
I highly suspect it is introduced by hive driver, since it appears quite recently, within a few days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] doglex commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by GitBox <gi...@apache.org>.
doglex commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1377131374
same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Connect to Apache Hive endpoint in OCI BDS [superset]
Posted by "tuoluzhe8521 (via GitHub)" <gi...@apache.org>.
tuoluzhe8521 commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-2547824091
there are a good idea https://github.com/Usiel/PyHive/commit/fb692d956758bca9d4aba980a8c418cc5788768d?diff=split
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Usiel commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "Usiel (via GitHub)" <gi...@apache.org>.
Usiel commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1592541855
Once PyHive releases the fix we can update the dependency on Superset. Currently, 0.7.0 is a pre-release: https://pypi.org/project/PyHive/#history
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Usiel commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "Usiel (via GitHub)" <gi...@apache.org>.
Usiel commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1575913646
> > The issue is related to this [newly added line](https://github.com/apache/superset/commit/e98943e5805fd23c4b3018bee342873f19ee6546#diff-462c756528595a2be4a0a3c3063077f984384c7858ecdb5bd3e9603163c3678bR251) and the fact that [PyHive returns bytes](https://github.com/dropbox/PyHive/blob/master/pyhive/sqlalchemy_hive.py#L232) instead of a string for the dialect's driver field, which then in turn messes with the JSON serialization. I figure we can handle this oddness on our side if needed, but I opened a PR in the meantime ([dropbox/PyHive#450](https://github.com/dropbox/PyHive/pull/450)).
> > A **less than ideal** workaround that might work: Use http to connect to your Hive server with or instead of the default thrift driver.`` hive+http://``hive+https:// ``
>
> it doesn't work for me. I see error:
>
> ```
> An error occurred while creating databases: (builtins.NoneType) None
> [SQL: Authentication is not valid use one of:BASIC, NOSASL, KERBEROS, NONE]
> (Background on this error at: https://sqlalche.me/e/14/dbapi)
> ```
Hi @welljs, seems unrelated to this issue to me. Probably a problem with the used SQLAlchemy URI (`auth` param?).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rusackas commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by GitBox <gi...@apache.org>.
rusackas commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1335797802
@bkyryliuk uses Hive, if i'm not mistaken. Have you run into anything like this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] ljyf5593 commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "ljyf5593 (via GitHub)" <gi...@apache.org>.
ljyf5593 commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1447351791
same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] vungle-wesleygoi commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "vungle-wesleygoi (via GitHub)" <gi...@apache.org>.
vungle-wesleygoi commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1592530301
need to be fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] terrancesnyder commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "terrancesnyder (via GitHub)" <gi...@apache.org>.
terrancesnyder commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1646827518
+1 please fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Usiel commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "Usiel (via GitHub)" <gi...@apache.org>.
Usiel commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1527594416
The issue is related to this [newly added line](https://github.com/apache/superset/commit/e98943e5805fd23c4b3018bee342873f19ee6546#diff-462c756528595a2be4a0a3c3063077f984384c7858ecdb5bd3e9603163c3678bR251) and the fact that [PyHive returns bytes](https://github.com/dropbox/PyHive/blob/master/pyhive/sqlalchemy_hive.py#L232) instead of a string for the dialect's driver field, which then in turn messes with the JSON serialization. I figure we can handle this oddness on our side if needed, but I opened a PR in the meantime (https://github.com/dropbox/PyHive/pull/450).
A **less than ideal** workaround that might work: Use http to connect to your Hive server with `hive+http://` or `hive+https://` instead of the default thrift driver.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] welljs commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "welljs (via GitHub)" <gi...@apache.org>.
welljs commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1566983687
> The issue is related to this [newly added line](https://github.com/apache/superset/commit/e98943e5805fd23c4b3018bee342873f19ee6546#diff-462c756528595a2be4a0a3c3063077f984384c7858ecdb5bd3e9603163c3678bR251) and the fact that [PyHive returns bytes](https://github.com/dropbox/PyHive/blob/master/pyhive/sqlalchemy_hive.py#L232) instead of a string for the dialect's driver field, which then in turn messes with the JSON serialization. I figure we can handle this oddness on our side if needed, but I opened a PR in the meantime ([dropbox/PyHive#450](https://github.com/dropbox/PyHive/pull/450)).
>
> A **less than ideal** workaround that might work: Use http to connect to your Hive server with or instead of the default thrift driver.`hive+http://``hive+https://`
it doesn't work for me. I see error:
```
An error occurred while creating databases: (builtins.NoneType) None
[SQL: Authentication is not valid use one of:BASIC, NOSASL, KERBEROS, NONE]
(Background on this error at: https://sqlalche.me/e/14/dbapi)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] bkyryliuk commented on issue #22316: Connect to Apache Hive endpoint in OCI BDS
Posted by "bkyryliuk (via GitHub)" <gi...@apache.org>.
bkyryliuk commented on issue #22316:
URL: https://github.com/apache/superset/issues/22316#issuecomment-1691954808
0.7.0 is released, give it a try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org