You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Rajesh Balamohan (Jira)" <ji...@apache.org> on 2020/10/20 09:55:00 UTC

[jira] [Created] (HIVE-24290) Explain analyze can be slow in cloud storage

Rajesh Balamohan created HIVE-24290:
---------------------------------------

             Summary: Explain analyze can be slow in cloud storage
                 Key: HIVE-24290
                 URL: https://issues.apache.org/jira/browse/HIVE-24290
             Project: Hive
          Issue Type: Improvement
            Reporter: Rajesh Balamohan


"explain analyze" takes a lot longer time to exit with the following path, specifically in cloud environments (where it could be EFS vol).  HIVE-24270 is a related ticket as well.

 
{noformat}
        at java.io.UnixFileSystem.delete0(Native Method)
	at java.io.UnixFileSystem.delete(UnixFileSystem.java:265)
	at java.io.File.delete(File.java:1043)
	at org.apache.hadoop.fs.FileUtil.deleteImpl(FileUtil.java:229)
	at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:270)
	at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:182)
	at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:153)
	at org.apache.hadoop.fs.RawLocalFileSystem.delete(RawLocalFileSystem.java:453)
	at org.apache.hadoop.fs.ChecksumFileSystem.delete(ChecksumFileSystem.java:685)
	at org.apache.hadoop.hive.ql.stats.fs.FSStatsAggregator.closeConnection(FSStatsAggregator.java:115)
	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.aggregateStats(ExplainSemanticAnalyzer.java:261)
	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:156)
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
	at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221)
	at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188)
	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:600)
	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:546)
	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:540)
	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127)

 {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)