[jira] [Commented] (HIVE-12348) Byte array comparison optimization for SIMD

["Hive QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-12348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15003431#comment-15003431 ] 

Hive QA commented on HIVE-12348:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12771988/HIVE-12348.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 9767 tests executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver-script_pipe.q-tez_smb_empty.q-mapjoin_decimal.q-and-12-more - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_rp_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.hwi.TestHWISessionManager.testHiveDriver
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6018/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6018/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6018/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12771988 - PreCommit-HIVE-TRUNK-Build

> Byte array comparison optimization for SIMD
> -------------------------------------------
>
>                 Key: HIVE-12348
>                 URL: https://issues.apache.org/jira/browse/HIVE-12348
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Teddy Choi
>            Assignee: Teddy Choi
>            Priority: Minor
>         Attachments: HIVE-12348.patch
>
>
> The current byte comparison implementation in Hive is basic. It handles a byte (1 byte) at a time, so it's slow.
> There's FastByteComparisons class in Hadoop with sun.misc.Unsafe class. (https://github.com/hanborq/hadoop/blob/master/src/core/org/apache/hadoop/io/FastByteComparisons.java) It handles a long integer (8 bytes) at a time, so it's faster.
> Java 8 has String.compare, String.equalTo intrinsics with AVX2 and SSE4.2. (http://hg.openjdk.java.net/jdk8/jdk8/hotspot/rev/038dd2875b94) It handles 128~256 bits (16~32 bytes) at a time, so it's much faster.
> However, Unsafe.getLong and String.compare intrinsic needs additional data copies, so the actual performance increase is smaller than "1 byte : 32 bytes" comparison.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)