You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jun He (JIRA)" <ji...@apache.org> on 2017/10/11 05:09:00 UTC
[jira] [Created] (MAPREDUCE-6980) Add support of ARM64 hardware crc
instructions for nativetask
Jun He created MAPREDUCE-6980:
---------------------------------
Summary: Add support of ARM64 hardware crc instructions for nativetask
Key: MAPREDUCE-6980
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6980
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: nativetask, performance
Environment: ARM64 platform with CRC extension
Reporter: Jun He
Priority: Minor
CRC instructions are introduced since ARM64V8 ISA. With this extension Arm supports CRC32(0x04C11DB7) and CRC32C(0x1EDC6F41) checksum hardware computation.
This patch enables support on such arm64 platform, similiar as [HADOOP-11660|https://issues.apache.org/jira/browse/HADOOP-11660].
Benchmark is done on a A57 platform using nttest (hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest), and the result is as following:
Before (sw crc):
[ RUN ] Perf.CRC
17/10/10 03:05:18 INFO CRC time: 0.10112s size: 100.000M speed: 988.92M/s
17/10/10 03:05:18 INFO CRC32C time: 0.10043s size: 100.000M speed: 995.74M/s
[ OK ] Perf.CRC (216 ms)
After (hw crc):
[ RUN ] Perf.CRC
17/10/11 05:06:27 INFO CRC time: 0.03173s size: 100.000M speed: 3151.14M/s
17/10/11 05:06:27 INFO CRC32C time: 0.03146s size: 100.000M speed: 3178.97M/s
[ OK ] Perf.CRC (77 ms)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org