[GitHub] [spark] mgaido91 commented on issue #24636: [SPARK-27684][SQL] Avoid conversion overhead for primitive types

[GitBox <gi...@apache.org>]

mgaido91 commented on issue #24636: [SPARK-27684][SQL] Avoid conversion overhead for primitive types
URL: https://github.com/apache/spark/pull/24636#issuecomment-494319192
 
 
   Thanks for you comment @kiszk .
   
   I added the warm-up, and the result is barely the same. Here is the code:
   ```
   test("SPARK-27684: perf evaluation") {
       val intLongUdf = ScalaUDF(
         (a: Int, b: Long) => a + b, LongType,
         Literal(1) :: Literal(1L) :: Nil,
         true :: true :: Nil,
         nullable = false)
   
       val plan = generateProject(
         MutableProjection.create(Alias(intLongUdf, s"udf")() :: Nil),
         intLongUdf)
       plan.initialize(0)
   
       var i = 0
       val N = 100000000
       while(i < 1000) {
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         i += 1
       }
       i = 0
       val t0 = System.nanoTime()
       while(i < N) {
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         plan(EmptyRow).get(0, intLongUdf.dataType)
         i += 1
       }
       val t1 = System.nanoTime()
       println(s"Avg time: ${(t1 - t0).toDouble / N} ns")
     }
   ```
   
   and the results are:
   ```
   Old Avg time: 49.58303799 ns
   New Avg time: 12.66588096 ns
   ```
   
   Any more comments/concerns?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org