[jira] [Created] (SPARK-12075) Speed up HiveComparisionTest suites by speeding up / avoiding reset()

["Josh Rosen (JIRA)" <ji...@apache.org>]

Josh Rosen created SPARK-12075:
----------------------------------

             Summary: Speed up HiveComparisionTest suites by speeding up / avoiding reset()
                 Key: SPARK-12075
                 URL: https://issues.apache.org/jira/browse/SPARK-12075
             Project: Spark
          Issue Type: Improvement
          Components: SQL, Tests
            Reporter: Josh Rosen
            Assignee: Josh Rosen


When profiling HiveCompatibilitySuite, I noticed that most of the time seems to be spent in expensive TestHive.reset() calls. We can speed up these tests by avoiding the TestHive.reset() calls when possible and by speeding up the reset() call itself by being more selective about when to load the {{src}} and {{srcpart}} tables. This can lead to a 10-15 minute test speedup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org