You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2019/10/17 11:13:00 UTC
[jira] [Commented] (HADOOP-16601) Add support for hardware crc32 of
nativetask checksums on aarch64 arch
[ https://issues.apache.org/jira/browse/HADOOP-16601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953630#comment-16953630 ]
Steve Loughran commented on HADOOP-16601:
-----------------------------------------
* it's best if you file a github PR with this JIRA at the start of the title
* Look at the Yetus complaints about style, swap tabs for spaces
we really love unit tests, and new patches to improve that coverage
* you MUST run the native tests on arm64 and give us the results; yetus won't be testing your code
* and if you can think of new tests to put into test_bulk_crc32.c, that'd be good too. Are there ways to break things?
> Add support for hardware crc32 of nativetask checksums on aarch64 arch
> ----------------------------------------------------------------------
>
> Key: HADOOP-16601
> URL: https://issues.apache.org/jira/browse/HADOOP-16601
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
> Affects Versions: 3.2.1
> Reporter: MacChen01
> Priority: Major
> Labels: performance
> Fix For: site
>
> Attachments: HADOOP-16601.patch
>
>
> Add support for aarch64 CRC instructions in nativetask module, optimize the CRC32 and CRC32C.
> Use the benchmark tools : nttest , the improvement is quite substantial:
> *CRC32 Zlib polynomial 0x04C11DB7*
> |KeyValueType-IO|Before(MB/s)|After(MB/s)|Improvement|
> |TextType-Write|425.98|602.92|+42%|
> |TextType-Read|796.06|1716.59|+116%|
> |BytesType-Write|474.25|686.84|+45%|
> |BytesType-Read|844.96|1955.03|+131%|
> |UnknownType-Write|434.84|608.81|+40%|
> |UnknownType-Read|805.76|1733.82|+115%|
>
>
> *CRC32C Castagnoli polynomial 0x1EDC6F41*
>
> |KeyValueType-IO|Before(MB/s)|After(MB/s)|Improvement|
> |TextType-Write|423.39|606.55|+43%|
> |TextType-Read|799.20|1783.28|+123%|
> |BytesType-Write|473.95|696.47|+47%|
> |BytesType-Read|846.30|2018.06|+138%|
> |UnknownType-Write|434.07|612.31|+41%|
> |UnknownType-Read|807.16|1783.95|+121%|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org