You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jun He (JIRA)" <ji...@apache.org> on 2017/10/16 02:03:00 UTC
[jira] [Updated] (MAPREDUCE-6980) Add support of ARM64 hardware crc
instructions for nativetask
[ https://issues.apache.org/jira/browse/MAPREDUCE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun He updated MAPREDUCE-6980:
------------------------------
Affects Version/s: 3.0.0-alpha3
Status: Patch Available (was: Open)
Implement ARM CRC instructions accelerated CRC computation.
Tested with nttest.
> Add support of ARM64 hardware crc instructions for nativetask
> -------------------------------------------------------------
>
> Key: MAPREDUCE-6980
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6980
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: nativetask, performance
> Affects Versions: 3.0.0-alpha3
> Environment: ARM64 platform with CRC extension
> Reporter: Jun He
> Priority: Minor
>
> CRC instructions are introduced since ARM64V8 ISA. With this extension Arm supports CRC32(0x04C11DB7) and CRC32C(0x1EDC6F41) checksum hardware computation.
> This patch enables support on such arm64 platform, similiar as [HADOOP-11660|https://issues.apache.org/jira/browse/HADOOP-11660].
> Benchmark is done on a A57 platform using nttest (hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest), and the result is as following:
> Before (sw crc):
> [ RUN ] Perf.CRC
> 17/10/10 03:05:18 INFO CRC time: 0.10112s size: 100.000M speed: 988.92M/s
> 17/10/10 03:05:18 INFO CRC32C time: 0.10043s size: 100.000M speed: 995.74M/s
> [ OK ] Perf.CRC (216 ms)
> After (hw crc):
> [ RUN ] Perf.CRC
> 17/10/11 05:06:27 INFO CRC time: 0.03173s size: 100.000M speed: 3151.14M/s
> 17/10/11 05:06:27 INFO CRC32C time: 0.03146s size: 100.000M speed: 3178.97M/s
> [ OK ] Perf.CRC (77 ms)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org