[jira] [Resolved] (SPARK-9460) Avoid byte array allocation in StringPrefixComparator

["Reynold Xin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-9460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Reynold Xin resolved SPARK-9460.
--------------------------------
       Resolution: Fixed
         Assignee: Reynold Xin
    Fix Version/s: 1.5.0

> Avoid byte array allocation in StringPrefixComparator
> -----------------------------------------------------
>
>                 Key: SPARK-9460
>                 URL: https://issues.apache.org/jira/browse/SPARK-9460
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Reynold Xin
>            Assignee: Reynold Xin
>             Fix For: 1.5.0
>
>
> StringPrefixComparator converts the long values back to byte arrays in order to compare them. We should be able to optimize this to compare the longs directly, rather than turning the longs into byte arrays and comparing them byte by byte. 
> {code}
>     public int compare(long aPrefix, long bPrefix) {
>       // TODO: can done more efficiently
>       byte[] a = Longs.toByteArray(aPrefix);
>       byte[] b = Longs.toByteArray(bPrefix);
>       for (int i = 0; i < 8; i++) {
>         int c = UnsignedBytes.compare(a[i], b[i]);
>         if (c != 0) return c;
>       }
>       return 0;
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org