You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/03/29 19:25:42 UTC

[GitHub] [superset] clarkperkins opened a new issue #19418: Databricks all-purpose clusters using pyhive driver broken in 1.4.0+

clarkperkins opened a new issue #19418:
URL: https://github.com/apache/superset/issues/19418


   In 1.4.0+, databases connecting to an all-purpose Databricks cluster with the databricks+pyhive url have duplicate entries in table dropdowns.
   
   #### How to reproduce the bug
   
   1. Add a database with a databricks+pyhive url
   2. Go to the sql editor (or try to create a dataset)
   3. Choose the databricks+pyhive database
   4. Choose a schema
   5. Click the "See Table Schema" dropdown
   
   ### Expected results
   
   No duplicate listing in the table dropdowns.
   
   ### Actual results
   
   There are two entries for every table in the table dropdowns (1 table and 1 view).
   Also, there is weirdness when scrolling through that dropdown list.
   
   
   ### Environment
   
   - browser type and version: MacOS Safari 15.4
   - superset version: 1.4.1
   - python version: 3.8.12 (from official apache/superset:1.4.1 docker image)
   - node.js version: not installed in the docker image
   - any feature flags active: none
   
   ### Checklist
   
   Make sure to follow these steps before submitting your issue - thank you!
   
   - [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
   - [x] I have reproduced the issue with at least the latest released version of superset.
   - [x] I have checked the issue tracker for the same issue and I haven't found one similar.
   
   ### Additional context
   
   The problem is caused by the way EngineSpecs are being loaded here: https://github.com/apache/superset/blob/master/superset/db_engine_specs/__init__.py#L81
   
   Only 1 EngineSpec is ever loaded for a given engine name - so databricks+pyhive urls end up using the ODBC EngineSpec, since it is defined last. I have proved this to be the case locally by commenting out the ODBC EngineSpec, and the issue resolves itself.
   
   If at all possible, it would be nice to resolve this issue by removing the 2 current databricks EngineSpecs and replacing with a single new spec that uses the non-deprecated official databricks connector described here: https://github.com/apache/superset/discussions/18552. 
   
   We have stubbed out this change and tried it out in our own environment, and would be happy to open a PR back to this repo, but weren't sure if it is ok to remove existing EngineSpecs.
   
   The new connector supports both all-purpose clusters and SQL endpoints out of the box, so there would be no need to have separate connectors.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org