You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/12/14 20:57:09 UTC
[GitHub] [superset] nagarajmmu opened a new issue, #22420: Superset failed to get Databricks tables
nagarajmmu opened a new issue, #22420:
URL: https://github.com/apache/superset/issues/22420
Hi Team
I have installed superset with docker container, logged in to superset.
I wanted to connect to Azure Databricks. I have followed the below steps.
I have created "requirements-local.txt" file in /superset/docker/ folder.
Added "databricks-dbapi[hive,odbc,sqlalchemy]" in "requirements-local.txt" file.
Ran "docker-compose build --force-rm" command to build.
Ram "docker-compose -f docker-compose.yml up" command to run the docker.
Superset is up and running and able to login.
Provided Databricks connection string to connection database.
**Got to create dataset, select DATABASE (Databricks SQL Endpoint), than select SCHEMA (any schema) ERROR Occurs**
#### How to reproduce the bug
1. Go to 'Datasets' menu
2. Click on +DATASET button to create a new dataset
3. Select DATABASE (Databricks SQL Endpoint) dropdown
4. Select SCHEMA (any schema) dropdown.
5. Error occurs.
### Expected results
All tables of selected schema to be available in table dropdown
### Actual results
ERROR:
There was an error loading the tables:
Traceback (most recent call last):
File "/app/superset/views/base.py", line 219, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 175, in wraps
return f(self, *args, **kwargs)
File "/app/superset/utils/log.py", line 265, in wrapper
value = f(*args, **kwargs)
File "/app/superset/views/core.py", line 1178, in tables
for datasource_name in database.get_all_table_names_in_schema(
File "/app/superset/utils/cache.py", line 131, in wrapped_f
return f(*args, **kwargs)
File "/app/superset/models/core.py", line 573, in get_all_table_names_in_schema
raise self.db_engine_spec.get_dbapi_mapped_exception(ex)
File "/app/superset/models/core.py", line 566, in get_all_table_names_in_schema
for table in self.db_engine_spec.get_table_names(
File "/app/superset/db_engine_specs/presto.py", line 430, in get_table_names
) - cls.get_view_names(database, inspector, schema)
File "/app/superset/db_engine_specs/presto.py", line 476, in get_view_names
cursor.execute(sql, params)
File "/usr/local/lib/python3.8/site-packages/pyhive/hive.py", line 455, in execute
_check_status(response)
File "/usr/local/lib/python3.8/site-packages/pyhive/hive.py", line 585, in _check_status
raise OperationalError(response)
pyhive.exc.OperationalError: TExecuteStatementResp(status=TStatus(statusCode=3, infoMessages=["*org.apache.hive.service.cli.HiveSQLException:Error running query: org.apache.spark.sql.AnalysisException: Table or view not found: information_schema.tables; line 1 pos 23;\n'Project ['table_name]\n+- 'Filter (('table_schema = default) AND ('table_type = VIEW))\n +- 'UnresolvedRelation [information_schema, tables]\n:58:57", 'org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation:org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute:SparkExecuteStatementOperation.scala:580', 'org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation:runInternal:SparkExecuteStatementOperation.scala:413', 'org.apache.hive.service.cli.operation.Operation:run:Operation.java:285', 'org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation:org$apache$spark$sql$hive$thriftserver$SparkOperation$$super$run:SparkExecuteStatementOperation.scala:49',
'org.apache.spark.sql.hive.thriftserver.SparkOperation:$anonfun$run$1:SparkOperation.scala:59', 'scala.runtime.java8.JFunction0$mcV$sp:apply:JFunction0$mcV$sp.java:23', 'org.apache.spark.sql.hive.thriftserver.SparkOperation:withLocalProperties:SparkOperation.scala:134', 'org.apache.spark.sql.hive.thriftserver.SparkOperation:withLocalProperties$:SparkOperation.scala:77', 'org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation:withLocalProperties:SparkExecuteStatementOperation.scala:49', 'org.apache.spark.sql.hive.thriftserver.SparkOperation:run:SparkOperation.scala:59', 'org.apache.spark.sql.hive.thriftserver.SparkOperation:run$:SparkOperation.scala:57', 'org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation:run:SparkExecuteStatementOperation.scala:49', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementInternal:HiveSessionImpl.java:543', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatement:HiveSessionImpl.java:519', '
org.apache.hive.service.cli.CLIService:executeStatement:CLIService.java:303', 'org.apache.hive.service.cli.thrift.ThriftCLIService:ExecuteStatement:ThriftCLIService.java:533', 'org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1437', 'org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1422', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:38', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.thrift.server.TServlet:doPost:TServlet.java:83', 'org.apache.hive.service.cli.thrift.ThriftHttpServlet:doPost:ThriftHttpServlet.java:195', 'javax.servlet.http.HttpServlet:service:HttpServlet.java:707', 'javax.servlet.http.HttpServlet:service:HttpServlet.java:790', 'org.eclipse.jetty.servlet.ServletHolder:handle:ServletHolder.java:873', 'org.eclipse.jetty.servlet.ServletHandler:doHandle:ServletHandler.java:542', 'org.eclipse.jetty.server.handler.Scoped
Handler:handle:ScopedHandler.java:146', 'org.eclipse.jetty.security.SecurityHandler:handle:SecurityHandler.java:548', 'org.eclipse.jetty.server.handler.HandlerWrapper:handle:HandlerWrapper.java:132', 'org.eclipse.jetty.server.handler.ScopedHandler:nextHandle:ScopedHandler.java:257', 'org.eclipse.jetty.server.session.SessionHandler:doHandle:SessionHandler.java:1700', 'org.eclipse.jetty.server.handler.ScopedHandler:nextHandle:ScopedHandler.java:255', 'org.eclipse.jetty.server.handler.ContextHandler:doHandle:ContextHandler.java:1345', 'org.eclipse.jetty.server.handler.ScopedHandler:nextScope:ScopedHandler.java:203', 'org.eclipse.jetty.servlet.ServletHandler:doScope:ServletHandler.java:480', 'org.eclipse.jetty.server.session.SessionHandler:doScope:SessionHandler.java:1667', 'org.eclipse.jetty.server.handler.ScopedHandler:nextScope:ScopedHandler.java:201', 'org.eclipse.jetty.server.handler.ContextHandler:doScope:ContextHandler.java:1247', 'org.eclipse.jetty.server.handler.ScopedHandler:h
andle:ScopedHandler.java:144', 'org.eclipse.jetty.server.handler.HandlerWrapper:handle:HandlerWrapper.java:132', 'org.eclipse.jetty.server.Server:handle:Server.java:505', 'org.eclipse.jetty.server.HttpChannel:handle:HttpChannel.java:370', 'org.eclipse.jetty.server.HttpConnection:onFillable:HttpConnection.java:267', 'org.eclipse.jetty.io.AbstractConnection$ReadCallback:succeeded:AbstractConnection.java:305', 'org.eclipse.jetty.io.FillInterest:fillable:FillInterest.java:103', 'org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint:onFillable:SslConnection.java:427', 'org.eclipse.jetty.io.ssl.SslConnection:onFillable:SslConnection.java:321', 'org.eclipse.jetty.io.ssl.SslConnection$2:succeeded:SslConnection.java:159', 'org.eclipse.jetty.io.FillInterest:fillable:FillInterest.java:103', 'org.eclipse.jetty.io.ChannelEndPoint$2:run:ChannelEndPoint.java:117', 'org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:runTask:EatWhatYouKill.java:333', 'org.eclipse.jetty.util.thread.strategy.Eat
WhatYouKill:doProduce:EatWhatYouKill.java:310', 'org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:tryProduce:EatWhatYouKill.java:168', 'org.eclipse.jetty.util.thread.strategy.EatWhatYouKill:run:EatWhatYouKill.java:126', 'org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread:run:ReservedThreadExecutor.java:366', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1149', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:624', 'java.lang.Thread:run:Thread.java:750', "*org.apache.spark.sql.AnalysisException:Table or view not found: information_schema.tables; line 1 pos 23;\n'Project ['table_name]\n+- 'Filter (('table_schema = default) AND ('table_type = VIEW))\n +- 'UnresolvedRelation [information_schema, tables]\n:93:35", 'org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt:failAnalysis:package.scala:42', 'org.apache.spark.sql.catalyst.analysis.CheckAnalysis:$anonfun$checkAnalysis$1:CheckAnalysis.scala:109
', 'org.apache.spark.sql.catalyst.analysis.CheckAnalysis:$anonfun$checkAnalysis$1$adapted:CheckAnalysis.scala:95', 'org.apache.spark.sql.catalyst.trees.TreeNode:foreachUp:TreeNode.scala:192', 'org.apache.spark.sql.catalyst.trees.TreeNode:$anonfun$foreachUp$1:TreeNode.scala:191', 'org.apache.spark.sql.catalyst.trees.TreeNode:$anonfun$foreachUp$1$adapted:TreeNode.scala:191', 'scala.collection.immutable.List:foreach:List.scala:392', 'org.apache.spark.sql.catalyst.trees.TreeNode:foreachUp:TreeNode.scala:191', 'org.apache.spark.sql.catalyst.trees.TreeNode:$anonfun$foreachUp$1:TreeNode.scala:191', 'org.apache.spark.sql.catalyst.trees.TreeNode:$anonfun$foreachUp$1$adapted:TreeNode.scala:191', 'scala.collection.immutable.List:foreach:List.scala:392', 'org.apache.spark.sql.catalyst.trees.TreeNode:foreachUp:TreeNode.scala:191', 'org.apache.spark.sql.catalyst.analysis.CheckAnalysis:checkAnalysis:CheckAnalysis.scala:95', 'org.apache.spark.sql.catalyst.analysis.CheckAnalysis:checkAnalysis$:Check
Analysis.scala:92', 'org.apache.spark.sql.catalyst.analysis.Analyzer:checkAnalysis:Analyzer.scala:180', 'org.apache.spark.sql.catalyst.analysis.Analyzer:$anonfun$executeAndCheck$1:Analyzer.scala:203', 'org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$:markInAnalyzer:AnalysisHelper.scala:223', 'org.apache.spark.sql.catalyst.analysis.Analyzer:executeAndCheck:Analyzer.scala:200', 'org.apache.spark.sql.execution.QueryExecution:$anonfun$analyzed$1:QueryExecution.scala:91', 'org.apache.spark.sql.catalyst.QueryPlanningTracker:measurePhase:QueryPlanningTracker.scala:111', 'org.apache.spark.sql.execution.QueryExecution:$anonfun$executePhase$1:QueryExecution.scala:171', 'org.apache.spark.sql.SparkSession:withActive:SparkSession.scala:845', 'org.apache.spark.sql.execution.QueryExecution:executePhase:QueryExecution.scala:171', 'org.apache.spark.sql.execution.QueryExecution:analyzed$lzycompute:QueryExecution.scala:92', 'org.apache.spark.sql.execution.QueryExecution:analyzed:QueryExecut
ion.scala:89', 'org.apache.spark.sql.execution.QueryExecution:assertAnalyzed:QueryExecution.scala:81', 'org.apache.spark.sql.Dataset$:$anonfun$ofRows$2:Dataset.scala:103', 'org.apache.spark.sql.SparkSession:withActive:SparkSession.scala:845', 'org.apache.spark.sql.Dataset$:ofRows:Dataset.scala:101', 'org.apache.spark.sql.SparkSession:$anonfun$sql$1:SparkSession.scala:680', 'org.apache.spark.sql.SparkSession:withActive:SparkSession.scala:845', 'org.apache.spark.sql.SparkSession:sql:SparkSession.scala:675', 'org.apache.spark.sql.SQLContext:sql:SQLContext.scala:672', 'org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation:$anonfun$execute$4:SparkExecuteStatementOperation.scala:514', 'scala.Option:getOrElse:Option.scala:189', 'org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation:org$apache$spark$sql$hive$thriftserver$SparkExecuteStatementOperation$$execute:SparkExecuteStatementOperation.scala:514'], sqlState=None, errorCode=0, errorMessage="Error running
query: org.apache.spark.sql.AnalysisException: Table or view not found: information_schema.tables; line 1 pos 23;\n'Project ['table_name]\n+- 'Filter (('table_schema = default) AND ('table_type = VIEW))\n +- 'UnresolvedRelation [information_schema, tables]\n"), operationHandle=None)
#### Screenshots
### Environment
- browser type and version: Chrome latest version
- superset version: 0.0.0-dev (Latest)
- python version: 3.8
- any feature flags active: DASHBOARD_CROSS_FILTER
Please let me know, how can I get Databricks table names in superset.
Thanks
Nagaraj M M
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Superset failed to get Databricks tables [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #22420: Superset failed to get Databricks tables
URL: https://github.com/apache/superset/issues/22420
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Superset failed to get Databricks tables [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #22420:
URL: https://github.com/apache/superset/issues/22420#issuecomment-1977255607
I hope that the answer above was sufficient to close the issue. I'll do so now since this one has been quiet for so long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] shenyubin1990 commented on issue #22420: Superset failed to get Databricks tables
Posted by "shenyubin1990 (via GitHub)" <gi...@apache.org>.
shenyubin1990 commented on issue #22420:
URL: https://github.com/apache/superset/issues/22420#issuecomment-1588591540
Hi Nagaraj,
I found the same issue like you described. I think you can try to run select current_catalog() to see if the catalog is the one you want to use. In my case, it will point to the first catalog in your sql warehouse even if you assigned the different database name. Unfortunately I haven't found a way to fix it, but I think that is a good start to dive deep.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org