You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jun He (JIRA)" <ji...@apache.org> on 2017/10/16 02:03:00 UTC

[jira] [Updated] (MAPREDUCE-6980) Add support of ARM64 hardware crc instructions for nativetask

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun He updated MAPREDUCE-6980:
------------------------------
    Affects Version/s: 3.0.0-alpha3
               Status: Patch Available  (was: Open)

Implement ARM CRC instructions accelerated CRC computation.
Tested with nttest.

> Add support of ARM64 hardware crc instructions for nativetask
> -------------------------------------------------------------
>
>                 Key: MAPREDUCE-6980
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6980
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: nativetask, performance
>    Affects Versions: 3.0.0-alpha3
>         Environment: ARM64 platform with CRC extension
>            Reporter: Jun He
>            Priority: Minor
>
> CRC instructions are introduced since ARM64V8 ISA. With this extension Arm supports CRC32(0x04C11DB7) and CRC32C(0x1EDC6F41) checksum hardware computation.
> This patch enables support on such arm64 platform, similiar as [HADOOP-11660|https://issues.apache.org/jira/browse/HADOOP-11660].
> Benchmark is done on a A57 platform using nttest (hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest), and the result is as following:
> Before (sw crc):
> [ RUN      ] Perf.CRC
> 17/10/10 03:05:18 INFO CRC time: 0.10112s size: 100.000M speed: 988.92M/s
> 17/10/10 03:05:18 INFO CRC32C time: 0.10043s size: 100.000M speed: 995.74M/s
> [       OK ] Perf.CRC (216 ms)
> After (hw crc):
> [ RUN      ] Perf.CRC
> 17/10/11 05:06:27 INFO CRC time: 0.03173s size: 100.000M speed: 3151.14M/s
> 17/10/11 05:06:27 INFO CRC32C time: 0.03146s size: 100.000M speed: 3178.97M/s
> [       OK ] Perf.CRC (77 ms)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org