You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2015/12/02 00:31:11 UTC
[jira] [Resolved] (SPARK-12075) Speed up HiveComparisionTest suites
by speeding up / avoiding reset()
[ https://issues.apache.org/jira/browse/SPARK-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reynold Xin resolved SPARK-12075.
---------------------------------
Resolution: Fixed
Fix Version/s: 1.6.0
> Speed up HiveComparisionTest suites by speeding up / avoiding reset()
> ---------------------------------------------------------------------
>
> Key: SPARK-12075
> URL: https://issues.apache.org/jira/browse/SPARK-12075
> Project: Spark
> Issue Type: Sub-task
> Components: SQL, Tests
> Reporter: Josh Rosen
> Assignee: Josh Rosen
> Fix For: 1.6.0
>
>
> When profiling HiveCompatibilitySuite, I noticed that most of the time seems to be spent in expensive TestHive.reset() calls. We can speed up these tests by avoiding the TestHive.reset() calls when possible and by speeding up the reset() call itself by being more selective about when to load the {{src}} and {{srcpart}} tables. This can lead to a 10-15 minute test speedup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org