You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/06/08 22:35:53 UTC

[GitHub] [spark] dongjoon-hyun commented on pull request #32826: [SPARK-35670][BUILD] Upgrade ZSTD-JNI to 1.5.0-1

dongjoon-hyun commented on pull request #32826:
URL: https://github.com/apache/spark/pull/32826#issuecomment-857226630


   BTW, @dchristle . I quickly ran the micro benchmark on master branch on my Mac before and after. We also need to check the memory usage, too.
   
   ```
    OpenJDK 64-Bit Server VM 1.8.0_292-b09 on Mac OS X 10.16
    Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Benchmark ZStandardCompressionCodec:                    Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
    --------------------------------------------------------------------------------------------------------------------------------------
   -Compression 10000 times at level 1 without buffer pool            173            348         161          0.1       17267.7       1.0X
   -Compression 10000 times at level 2 without buffer pool            616            669          57          0.0       61574.1       0.3X
   -Compression 10000 times at level 3 without buffer pool           1302           1327          35          0.0      130234.4       0.1X
   -Compression 10000 times at level 1 with buffer pool               177            192           9          0.1       17709.8       1.0X
   -Compression 10000 times at level 2 with buffer pool               670            709          36          0.0       66965.9       0.3X
   -Compression 10000 times at level 3 with buffer pool              1201           1209          11          0.0      120144.2       0.1X
   +Compression 10000 times at level 1 without buffer pool            271            348          75          0.0       27106.5       1.0X
   +Compression 10000 times at level 2 without buffer pool            655            720          59          0.0       65510.4       0.4X
   +Compression 10000 times at level 3 without buffer pool            908            963          73          0.0       90777.5       0.3X
   +Compression 10000 times at level 1 with buffer pool               181            194          11          0.1       18089.8       1.5X
   +Compression 10000 times at level 2 with buffer pool               489            531          48          0.0       48917.8       0.6X
   +Compression 10000 times at level 3 with buffer pool               859            948         108          0.0       85869.0       0.3X
   
    OpenJDK 64-Bit Server VM 1.8.0_292-b09 on Mac OS X 10.16
    Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Benchmark ZStandardCompressionCodec:                        Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
    ------------------------------------------------------------------------------------------------------------------------------------------
   -Decompression 10000 times from level 1 without buffer pool            422            441          12          0.0       42239.7       1.0X
   -Decompression 10000 times from level 2 without buffer pool            433            460          19          0.0       43342.9       1.0X
   -Decompression 10000 times from level 3 without buffer pool           1241           1306          93          0.0      124091.5       0.3X
   -Decompression 10000 times from level 1 with buffer pool               373            387          12          0.0       37268.1       1.1X
   -Decompression 10000 times from level 2 with buffer pool               383            387           3          0.0       38296.0       1.1X
   -Decompression 10000 times from level 3 with buffer pool              1116           1168          73          0.0      111603.1       0.4X
   +Decompression 10000 times from level 1 without buffer pool            464            487          21          0.0       46420.0       1.0X
   +Decompression 10000 times from level 2 without buffer pool            436            452          11          0.0       43596.2       1.1X
   +Decompression 10000 times from level 3 without buffer pool           1308           1345          53          0.0      130781.6       0.4X
   +Decompression 10000 times from level 1 with buffer pool               379            392          11          0.0       37895.4       1.2X
   +Decompression 10000 times from level 2 with buffer pool               380            396          12          0.0       38038.2       1.2X
   +Decompression 10000 times from level 3 with buffer pool               909           1001         130          0.0       90861.3       0.5X
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org