[ https://issues.apache.org/jira/browse/HADOOP-10865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14636436#comment-14636436 ]
Tsuyoshi Ozawa commented on HADOOP-10865:
-----------------------------------------
{quote}
$ bin/hadoop org.apache.hadoop.util.Crc32PerformanceTest
{quote}
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 32 | 1 | 329.3 | 685.5 | 108.1% | 1383.0 | 319.9% | 101.7% |
| 32 | 2 | 329.8 | 685.5 | 107.8% | 1382.4 | 319.1% | 101.7% |
| 32 | 4 | 251.2 | 627.1 | 149.7% | 1099.4 | 337.7% | 75.3% |
| 32 | 8 | 291.2 | 494.2 | 69.7% | 1368.9 | 370.1% | 177.0% |
| 32 | 16 | 149.3 | 290.4 | 94.5% | 814.6 | 445.5% | 180.5% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 64 | 1 | 474.1 | 816.5 | 72.2% | 1444.6 | 204.7% | 76.9% |
| 64 | 2 | 470.3 | 817.3 | 73.8% | 1443.9 | 207.0% | 76.7% |
| 64 | 4 | 471.2 | 816.6 | 73.3% | 1441.2 | 205.8% | 76.5% |
| 64 | 8 | 471.3 | 813.8 | 72.7% | 1430.1 | 203.5% | 75.7% |
| 64 | 16 | 280.2 | 423.0 | 51.0% | 959.8 | 242.6% | 126.9% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 128 | 1 | 603.4 | 900.6 | 49.3% | 1435.7 | 137.9% | 59.4% |
| 128 | 2 | 604.8 | 901.5 | 49.1% | 1448.7 | 139.5% | 60.7% |
| 128 | 4 | 604.2 | 900.5 | 49.0% | 1446.9 | 139.5% | 60.7% |
| 128 | 8 | 582.3 | 895.3 | 53.7% | 1343.9 | 130.8% | 50.1% |
| 128 | 16 | 412.9 | 493.2 | 19.4% | 1064.6 | 157.8% | 115.9% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 256 | 1 | 701.9 | 949.1 | 35.2% | 1448.2 | 106.3% | 52.6% |
| 256 | 2 | 702.8 | 950.9 | 35.3% | 1448.5 | 106.1% | 52.3% |
| 256 | 4 | 697.4 | 949.4 | 36.1% | 1445.9 | 107.3% | 52.3% |
| 256 | 8 | 682.6 | 845.0 | 23.8% | 1435.3 | 110.3% | 69.9% |
| 256 | 16 | 528.1 | 536.5 | 1.6% | 1113.2 | 110.8% | 107.5% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 512 | 1 | 757.1 | 971.9 | 28.4% | 1437.9 | 89.9% | 47.9% |
| 512 | 2 | 731.0 | 921.2 | 26.0% | 1437.3 | 96.6% | 56.0% |
| 512 | 4 | 708.1 | 890.9 | 25.8% | 1435.6 | 102.7% | 61.1% |
| 512 | 8 | 678.6 | 880.4 | 29.7% | 1424.5 | 109.9% | 61.8% |
| 512 | 16 | 594.3 | 555.5 | -6.5% | 1146.5 | 92.9% | 106.4% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 1024 | 1 | 776.0 | 954.8 | 23.0% | 1444.5 | 86.1% | 51.3% |
| 1024 | 2 | 774.9 | 943.5 | 21.8% | 1444.3 | 86.4% | 53.1% |
| 1024 | 4 | 772.8 | 948.1 | 22.7% | 1442.4 | 86.6% | 52.1% |
| 1024 | 8 | 757.5 | 929.2 | 22.7% | 1431.4 | 89.0% | 54.0% |
| 1024 | 16 | 654.2 | 581.1 | -11.2% | 1159.8 | 77.3% | 99.6% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 2048 | 1 | 792.5 | 955.4 | 20.6% | 1448.6 | 82.8% | 51.6% |
| 2048 | 2 | 793.5 | 949.7 | 19.7% | 1447.9 | 82.5% | 52.5% |
| 2048 | 4 | 787.1 | 776.8 | -1.3% | 1316.2 | 67.2% | 69.4% |
| 2048 | 8 | 762.9 | 944.1 | 23.7% | 1435.5 | 88.2% | 52.1% |
| 2048 | 16 | 693.9 | 594.8 | -14.3% | 1174.0 | 69.2% | 97.4% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 4096 | 1 | 800.7 | 959.2 | 19.8% | 1450.6 | 81.2% | 51.2% |
| 4096 | 2 | 800.2 | 958.6 | 19.8% | 1450.3 | 81.2% | 51.3% |
| 4096 | 4 | 801.0 | 957.3 | 19.5% | 1448.0 | 80.8% | 51.3% |
| 4096 | 8 | 798.0 | 952.0 | 19.3% | 1437.3 | 80.1% | 51.0% |
| 4096 | 16 | 720.4 | 605.5 | -15.9% | 1176.9 | 63.4% | 94.4% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 8192 | 1 | 810.3 | 966.1 | 19.2% | 1451.3 | 79.1% | 50.2% |
| 8192 | 2 | 808.2 | 966.0 | 19.5% | 1450.8 | 79.5% | 50.2% |
| 8192 | 4 | 809.4 | 790.1 | -2.4% | 1323.2 | 63.5% | 67.5% |
| 8192 | 8 | 807.5 | 961.0 | 19.0% | 1437.7 | 78.1% | 49.6% |
| 8192 | 16 | 727.3 | 610.7 | -16.0% | 1186.4 | 63.1% | 94.3% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 16384 | 1 | 803.9 | 956.5 | 19.0% | 1451.8 | 80.6% | 51.8% |
| 16384 | 2 | 804.1 | 957.9 | 19.1% | 1451.3 | 80.5% | 51.5% |
| 16384 | 4 | 803.6 | 786.8 | -2.1% | 1449.2 | 80.3% | 84.2% |
| 16384 | 8 | 784.7 | 953.4 | 21.5% | 1375.1 | 75.2% | 44.2% |
| 16384 | 16 | 730.6 | 618.4 | -15.4% | 1181.0 | 61.7% | 91.0% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 32768 | 1 | 807.0 | 956.8 | 18.6% | 1451.8 | 79.9% | 51.7% |
| 32768 | 2 | 807.6 | 957.2 | 18.5% | 1451.6 | 79.8% | 51.6% |
| 32768 | 4 | 806.8 | 958.0 | 18.7% | 1449.7 | 79.7% | 51.3% |
| 32768 | 8 | 803.5 | 954.0 | 18.7% | 1438.7 | 79.1% | 50.8% |
| 32768 | 16 | 732.4 | 621.3 | -15.2% | 1183.8 | 61.6% | 90.5% |
| bpc | #T || Zip || PureJava | % diff || Native | % diff | % diff |
| 65536 | 1 | 807.4 | 957.2 | 18.5% | 1452.5 | 79.9% | 51.8% |
| 65536 | 2 | 808.4 | 958.4 | 18.6% | 1451.6 | 79.6% | 51.5% |
| 65536 | 4 | 807.7 | 958.5 | 18.7% | 1449.7 | 79.5% | 51.3% |
| 65536 | 8 | 805.5 | 954.8 | 18.5% | 1439.1 | 78.7% | 50.7% |
| 65536 | 16 | 731.8 | 620.2 | -15.3% | 1185.9 | 62.0% | 91.2% |
> Add a Crc32 chunked verification benchmark for both directly and non-directly buffer cases
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-10865
> URL: https://issues.apache.org/jira/browse/HADOOP-10865
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Priority: Minor
> Labels: BB2015-05-TBR
> Attachments: HADOOP-10865.002.patch, c10865_20140717.patch
>
>
> Currently, it is not easy to compare Crc32 chunked verification implementations. Let's add a benchmark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)