You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2015/12/01 21:37:11 UTC
[jira] [Assigned] (SPARK-12075) Speed up HiveComparisionTest suites
by speeding up / avoiding reset()
[ https://issues.apache.org/jira/browse/SPARK-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-12075:
------------------------------------
Assignee: Apache Spark (was: Josh Rosen)
> Speed up HiveComparisionTest suites by speeding up / avoiding reset()
> ---------------------------------------------------------------------
>
> Key: SPARK-12075
> URL: https://issues.apache.org/jira/browse/SPARK-12075
> Project: Spark
> Issue Type: Improvement
> Components: SQL, Tests
> Reporter: Josh Rosen
> Assignee: Apache Spark
>
> When profiling HiveCompatibilitySuite, I noticed that most of the time seems to be spent in expensive TestHive.reset() calls. We can speed up these tests by avoiding the TestHive.reset() calls when possible and by speeding up the reset() call itself by being more selective about when to load the {{src}} and {{srcpart}} tables. This can lead to a 10-15 minute test speedup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org