You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "sdfox (JIRA)" <ji...@apache.org> on 2015/04/03 16:07:54 UTC
[jira] [Commented] (SPARK-6241) hiveql ANALYZE TABLE doesn't work
for external tables
[ https://issues.apache.org/jira/browse/SPARK-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394455#comment-14394455 ]
sdfox commented on SPARK-6241:
------------------------------
Can you give a sample code to reproduce this problem?
> hiveql ANALYZE TABLE doesn't work for external tables
> -----------------------------------------------------
>
> Key: SPARK-6241
> URL: https://issues.apache.org/jira/browse/SPARK-6241
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.3.0
> Reporter: Kai Zeng
> Priority: Critical
>
> "ANALYZE TABLE" does not collect statistics for external tables, but works well for tables created by "CREATE AS SELECT".
> Also tried to use "refresh table" to refresh metadata cache, but got NullPointer error:
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
> at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
> at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
> at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:135)
> at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2344)
> at com.google.common.cache.LocalCache$Segment$1.run(LocalCache.java:2327)
> at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
> at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
> at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:101)
> at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:170)
> at com.google.common.cache.LocalCache$Segment.loadAsync(LocalCache.java:2322)
> at com.google.common.cache.LocalCache$Segment.refresh(LocalCache.java:2385)
> at com.google.common.cache.LocalCache.refresh(LocalCache.java:4085)
> at com.google.common.cache.LocalCache$LocalLoadingCache.refresh(LocalCache.java:4825)
> at org.apache.spark.sql.hive.HiveMetastoreCatalog.refreshTable(HiveMetastoreCatalog.scala:108)
> at org.apache.spark.sql.sources.RefreshTable.run(ddl.scala:404)
> at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult$lzycompute(commands.scala:55)
> at org.apache.spark.sql.execution.ExecutedCommand.sideEffectResult(commands.scala:55)
> at org.apache.spark.sql.execution.ExecutedCommand.execute(commands.scala:65)
> at org.apache.spark.sql.SQLContext$QueryExecution.toRdd$lzycompute(SQLContext.scala:1092)
> at org.apache.spark.sql.SQLContext$QueryExecution.toRdd(SQLContext.scala:1092)
> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:134)
> at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:117)
> at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:51)
> at org.apache.spark.sql.hive.HiveContext.sql(HiveContext.scala:92)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org