[jira] [Commented] (SPARK-12075) Speed up HiveComparisionTest suites by speeding up / avoiding reset()

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034527#comment-15034527 ] 

Apache Spark commented on SPARK-12075:
--------------------------------------

User 'JoshRosen' has created a pull request for this issue:
https://github.com/apache/spark/pull/10055

> Speed up HiveComparisionTest suites by speeding up / avoiding reset()
> ---------------------------------------------------------------------
>
>                 Key: SPARK-12075
>                 URL: https://issues.apache.org/jira/browse/SPARK-12075
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL, Tests
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> When profiling HiveCompatibilitySuite, I noticed that most of the time seems to be spent in expensive TestHive.reset() calls. We can speed up these tests by avoiding the TestHive.reset() calls when possible and by speeding up the reset() call itself by being more selective about when to load the {{src}} and {{srcpart}} tables. This can lead to a 10-15 minute test speedup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org