You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Tsz Wo Nicholas Sze (JIRA)" <ji...@apache.org> on 2014/06/10 00:17:01 UTC
[jira] [Commented] (HADOOP-10674) Rewrite the PureJavaCrc32 loop
for performance improvement
[ https://issues.apache.org/jira/browse/HADOOP-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025847#comment-14025847 ]
Tsz Wo Nicholas Sze commented on HADOOP-10674:
----------------------------------------------
java.version = 1.6.0_65
java.runtime.name = Java(TM) SE Runtime Environment
java.runtime.version = 1.6.0_65-b14-462-11M4609
java.vm.version = 20.65-b04-462
java.vm.vendor = Apple Inc.
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.version = 1.0
java.specification.version = 1.6
os.arch = x86_64
os.name = Mac OS X
os.version = 10.9.3
Performance Table (The unit is MB/sec)
|| Num Bytes || CRC32 || PureJavaCrc32 | % diff || PureJavaCrc32new | % diff ||
| 1 | 17.170 | 174.807 | 918.1% | 184.712 | 5.7% |
| 2 | 35.428 | 290.235 | 719.2% | 275.524 | -5.1% |
| 4 | 67.132 | 338.614 | 404.4% | 334.880 | -1.1% |
| 8 | 115.325 | 623.372 | 440.5% | 658.562 | 5.6% |
| 16 | 180.665 | 829.639 | 359.2% | 926.114 | 11.6% |
| 32 | 254.776 | 977.622 | 283.7% | 1186.548 | 21.4% |
| 64 | 315.420 | 1072.557 | 240.0% | 1292.799 | 20.5% |
| 128 | 360.204 | 1111.217 | 208.5% | 1483.043 | 33.5% |
| 256 | 390.422 | 1139.855 | 192.0% | 1557.234 | 36.6% |
| 512 | 404.637 | 1003.717 | 148.1% | 1590.339 | 58.4% |
| 1024 | 413.142 | 1167.351 | 182.6% | 1588.641 | 36.1% |
| 2048 | 413.260 | 1128.091 | 173.0% | 1630.324 | 44.5% |
| 4096 | 412.476 | 1145.957 | 177.8% | 1660.634 | 44.9% |
| 8192 | 416.169 | 1185.569 | 184.9% | 1600.820 | 35.0% |
| 16384 | 420.509 | 1181.968 | 181.1% | 1601.702 | 35.5% |
| 32768 | 418.372 | 1141.980 | 173.0% | 1610.671 | 41.0% |
| 65536 | 418.190 | 1146.984 | 174.3% | 1682.749 | 46.7% |
| 131072 | 415.032 | 1180.082 | 184.3% | 1571.806 | 33.2% |
| 262144 | 414.864 | 1186.768 | 186.1% | 1684.632 | 42.0% |
| 524288 | 421.510 | 1196.519 | 183.9% | 1580.774 | 32.1% |
| 1048576 | 422.179 | 1187.153 | 181.2% | 1628.685 | 37.2% |
| 2097152 | 422.650 | 1108.484 | 162.3% | 1642.879 | 48.2% |
| 4194304 | 418.723 | 1114.216 | 166.1% | 1663.223 | 49.3% |
| 8388608 | 419.063 | 1034.310 | 146.8% | 1631.987 | 57.8% |
| 16777216 | 421.176 | 1181.302 | 180.5% | 1636.996 | 38.6% |
> Rewrite the PureJavaCrc32 loop for performance improvement
> ----------------------------------------------------------
>
> Key: HADOOP-10674
> URL: https://issues.apache.org/jira/browse/HADOOP-10674
> Project: Hadoop Common
> Issue Type: Bug
> Components: performance, util
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
>
> Below are some performance improvement opportunities performance improvement in PureJavaCrc32.
> - eliminate "off += 8; len -= 8;"
> - replace T8_x_start with hard coded constants
> - eliminate c0 - c7 local variables
> In my machine, there are 30% to 50% improvement for most of the cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)