You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by cloud-fan <gi...@git.apache.org> on 2016/01/18 22:56:42 UTC

[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

GitHub user cloud-fan opened a pull request:

    https://github.com/apache/spark/pull/10816

    [SPARK-12888][SQL] benchmark the new hash expression

    Benchmark it on 4 different schemas, the result:
    ```
    Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
    Hash For simple:                   Avg Time(ms)    Avg Rate(M/s)  Relative Rate
    -------------------------------------------------------------------------------
    interpreted version                       31.47           266.54         1.00 X
    codegen version                           64.52           130.01         0.49 X
    ```
    
    ```
    Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
    Hash For normal:                   Avg Time(ms)    Avg Rate(M/s)  Relative Rate
    -------------------------------------------------------------------------------
    interpreted version                     4068.11             0.26         1.00 X
    codegen version                         1175.92             0.89         3.46 X
    ```
    
    ```
    Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
    Hash For array:                    Avg Time(ms)    Avg Rate(M/s)  Relative Rate
    -------------------------------------------------------------------------------
    interpreted version                     9276.70             0.06         1.00 X
    codegen version                        14762.23             0.04         0.63 X
    ```
    
    ```
    Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
    Hash For map:                      Avg Time(ms)    Avg Rate(M/s)  Relative Rate
    -------------------------------------------------------------------------------
    interpreted version                    58869.79             0.01         1.00 X
    codegen version                         9285.36             0.06         6.34 X
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cloud-fan/spark hash-benchmark

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/10816.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #10816
    
----
commit a8d9433b622c005390e82804f0c0c09c53cb965d
Author: Wenchen Fan <we...@databricks.com>
Date:   2016-01-16T05:21:09Z

    benchmark for hash expression

commit 28763a1bf7a70683ef392c3f7a7dbf5f56ec1fde
Author: Wenchen Fan <we...@databricks.com>
Date:   2016-01-17T21:10:58Z

    Merge remote-tracking branch 'origin/master' into hash-benchmark

commit d753ce8480115ffad37f8570b70c12a42d7e69b8
Author: Wenchen Fan <we...@databricks.com>
Date:   2016-01-18T21:07:02Z

    Merge remote-tracking branch 'origin/master' into hash-benchmark

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172666446
  
    **[Test build #49621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49621/consoleFull)** for PR 10816 at commit [`d753ce8`](https://github.com/apache/spark/commit/d753ce8480115ffad37f8570b70c12a42d7e69b8).
     * This patch **fails RAT tests**.
     * This patch merges cleanly.
     * This patch adds the following public classes _(experimental)_:
      * `class ALS(@Since(\"1.4.0\") override val uid: String) extends Estimator[ALSModel] with ALSParams`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172667569
  
    The hash exprssion is slow on simple schema because the `UnsafeProjection` has some unnecessary opeations every round and will hurt the performance if each round is quite small(like simple schema), this can be fixed by https://github.com/apache/spark/pull/10809. For array type, still investigate.
    
    cc @nongli 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172666454
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49621/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-173393934
  
    I'm going to merge this. Please submit an update once #10809 is merged and put the result in the test case as comments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172670697
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172666452
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172688101
  
    I'm confused by the conclusino. We don't use UnsafeProjection anymore right?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on a diff in the pull request:

    https://github.com/apache/spark/pull/10816#discussion_r50158415
  
    --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/HashBenchmark.scala ---
    @@ -0,0 +1,104 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.spark.sql
    +
    +import org.apache.spark.sql.catalyst.encoders.RowEncoder
    +import org.apache.spark.sql.catalyst.expressions.{Murmur3Hash, UnsafeProjection}
    +import org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection
    +import org.apache.spark.sql.types._
    +import org.apache.spark.util.Benchmark
    +
    +/**
    + * Benchmark for the previous interpreted hash function(InternalRow.hashCode) vs the new codegen
    + * hash expression(Murmur3Hash).
    + */
    +object HashBenchmark {
    +
    +  def test(name: String, schema: StructType, iters: Int): Unit = {
    +    val numRows = 1024 * 8
    +
    +    val generator = RandomDataGenerator.forType(schema, nullable = false).get
    +    val encoder = RowEncoder(schema)
    +    val attrs = schema.toAttributes
    +    val safeProjection = GenerateSafeProjection.generate(attrs, attrs)
    +
    +    val rows = (1 to numRows).map(_ =>
    +      // The output of encoder is UnsafeRow, use safeProjection to turn in into safe format.
    +      safeProjection(encoder.toRow(generator().asInstanceOf[Row])).copy()
    +    ).toArray
    +
    +    val benchmark = new Benchmark("Hash For " + name, iters * numRows)
    +    benchmark.addCase("interpreted version") { _: Int =>
    +      for (_ <- 0L until iters) {
    +        var sum = 0
    +        var i = 0
    +        while (i < numRows) {
    +          sum += rows(i).hashCode()
    +          i += 1
    +        }
    +      }
    +    }
    +
    +    val getHashCode = UnsafeProjection.create(new Murmur3Hash(attrs) :: Nil, attrs)
    +    benchmark.addCase("codegen version") { _: Int =>
    +      for (_ <- 0L until iters) {
    +        var sum = 0
    +        var i = 0
    +        while (i < numRows) {
    +          sum += getHashCode(rows(i)).getInt(0)
    +          i += 1
    +        }
    +      }
    +    }
    +    benchmark.run()
    +  }
    +
    +  def main(args: Array[String]): Unit = {
    +    val simple = new StructType().add("i", IntegerType)
    +    test("simple", simple, 1024)
    +
    +    val normal = new StructType()
    +      .add("null", NullType)
    +      .add("boolean", BooleanType)
    +      .add("byte", ByteType)
    +      .add("short", ShortType)
    +      .add("int", IntegerType)
    +      .add("long", LongType)
    +      .add("float", FloatType)
    +      .add("double", DoubleType)
    +      .add("bigDecimal", DecimalType.SYSTEM_DEFAULT)
    +      .add("smallDecimal", DecimalType.USER_DEFAULT)
    +      .add("string", StringType)
    +      .add("binary", BinaryType)
    +      .add("date", DateType)
    +      .add("timestamp", TimestampType)
    +    test("normal", normal, 128)
    +
    +    val arrayOfInt = ArrayType(IntegerType)
    +    val array = new StructType()
    +      .add("array", arrayOfInt)
    +      .add("arrayOfArray", ArrayType(arrayOfInt))
    --- End diff --
    
    Can you include the results as comments for each test() case?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172691721
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by nongli <gi...@git.apache.org>.
Github user nongli commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172954932
  
    LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172670878
  
    retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172674197
  
    **[Test build #49629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49629/consoleFull)** for PR 10816 at commit [`1771733`](https://github.com/apache/spark/commit/17717339c17360689deea80947729c7f8747e40a).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/10816


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172670700
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49625/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172691560
  
    **[Test build #49629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49629/consoleFull)** for PR 10816 at commit [`1771733`](https://github.com/apache/spark/commit/17717339c17360689deea80947729c7f8747e40a).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172956315
  
    we should run this benchmark again after https://github.com/apache/spark/pull/10809 is merged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172691722
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49629/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172689537
  
    We need `UnsafeProjection` to execute the codegen version of hash expression.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [SPARK-12888][SQL] benchmark the new hash expr...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/10816#issuecomment-172666327
  
    **[Test build #49621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49621/consoleFull)** for PR 10816 at commit [`d753ce8`](https://github.com/apache/spark/commit/d753ce8480115ffad37f8570b70c12a42d7e69b8).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org