[jira] [Commented] (ARROW-12046) [C++] Support string collation for sorting

["Benson Muite (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-12046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17427570#comment-17427570 ] 

Benson Muite commented on ARROW-12046:
--------------------------------------

Compression will be helpful.  One way of sorting Mandarin is by stroke count, for which there is [a table for unicode|https://www.unicode.org/reports/tr38/#SortingAlgorithm] However, this is not the only way so allowing developers freedom to create appropriate comparison functions is useful. [Minimal-icu-collation|https://github.com/Mytherin/minimal-icu-collation] also seems helpful, but is not vectorized. Javascript has a specification for [comparison functions|https://402.ecma-international.org/8.0/#sec-collator-compare-functions] - while there will be a loss of efficiency, something like this seems most appropriate. An example of using this can be found in this [DataTables blog|https://www.datatables.net/blog/2017-02-28]. [QString|https://doc.qt.io/qt-5/qstring.html] has a [locale aware comparison|https://doc.qt.io/qt-5/qstring.html#localeAwareCompare] but licensing only allows incorporation of Apache code into GPL or LGPL code, and not the other way round.

> [C++] Support string collation for sorting
> ------------------------------------------
>
>                 Key: ARROW-12046
>                 URL: https://issues.apache.org/jira/browse/ARROW-12046
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>    Affects Versions: 3.0.0
>            Reporter: Ian Cook
>            Priority: Major
>             Fix For: 7.0.0
>
>
> Currently the C++ library orders strings lexicographically as bytestrings. We should implement the capability to change string sorting behavior based on locale settings for string collation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)