[jira] [Created] (ARROW-11410) [Rust][Parquet] Implement returning dictionary arrays from parquet reader

["Yordan Pavlov (Jira)" <ji...@apache.org>]

Yordan Pavlov created ARROW-11410:
-------------------------------------

             Summary: [Rust][Parquet] Implement returning dictionary arrays from parquet reader
                 Key: ARROW-11410
                 URL: https://issues.apache.org/jira/browse/ARROW-11410
             Project: Apache Arrow
          Issue Type: Improvement
          Components: Rust
            Reporter: Yordan Pavlov


Currently the Rust parquet reader returns a regular array (e.g. string array) even when the column is dictionary encoded in the parquet file.

If the parquet reader had the ability to return dictionary arrays for dictionary encoded columns this would bring many benefits such as:
 * faster reading of dictionary encoded columns from parquet (as no conversion/expansion into a regular array would be necessary)
 * more efficient memory use as the dictionary array would use less memory when loaded in memory
 * faster filtering operations as SIMD can be used to filter over the numeric keys of a dictionary string array instead of comparing string values in a string array

[~nevime] , [~alamb]  let me know what you think



--
This message was sent by Atlassian Jira
(v8.3.4#803005)