You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2017/08/27 04:45:00 UTC
[jira] [Updated] (SPARK-21846) Reduce the number of shuffle
partition in Hive tests
[ https://issues.apache.org/jira/browse/SPARK-21846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-21846:
----------------------------------
Description:
Currently, Jenkins on master branch fails due to timeout.
According to the last successful run, Hive tests takes 50 mins from 132 mins (in Java/Scala tests)
- https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7/3396/testReport/
{code}
org.apache.spark.sql.hive.execution 30 min
org.apache.spark.sql.hive 11 min
org.apache.spark.sql.hive.client 5 min 2 sec
org.apache.spark.sql.hive.orc 1 min 25 sec
..
{code}
This issue aims to reduce the test time by reducing the number of shuffle partitions in Hive tests. This will reduce 10% of Hive test time.
was:
Currently, Jenkins on master branch fails due to timeout.
According to the last successful run, Hive tests takes 50 mins from 132 mins (in Java/Scala tests)
- https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7/3396/testReport/
{code}
org.apache.spark.sql.hive.execution 30 min
org.apache.spark.sql.hive 11 min
org.apache.spark.sql.hive.client 5 min 2 sec
org.apache.spark.sql.hive.orc 1 min 25 sec
..
{code}
This issue aims to reduce the test time by reducing the number of shuffle partitions in Hive tests.
> Reduce the number of shuffle partition in Hive tests
> ----------------------------------------------------
>
> Key: SPARK-21846
> URL: https://issues.apache.org/jira/browse/SPARK-21846
> Project: Spark
> Issue Type: Task
> Components: SQL, Tests
> Affects Versions: 2.3.0
> Reporter: Dongjoon Hyun
> Priority: Minor
>
> Currently, Jenkins on master branch fails due to timeout.
> According to the last successful run, Hive tests takes 50 mins from 132 mins (in Java/Scala tests)
> - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7/3396/testReport/
> {code}
> org.apache.spark.sql.hive.execution 30 min
> org.apache.spark.sql.hive 11 min
> org.apache.spark.sql.hive.client 5 min 2 sec
> org.apache.spark.sql.hive.orc 1 min 25 sec
> ..
> {code}
> This issue aims to reduce the test time by reducing the number of shuffle partitions in Hive tests. This will reduce 10% of Hive test time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org