You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@arrow.apache.org by Evan Chan <ev...@apple.com.INVALID> on 2020/03/10 04:07:07 UTC

[Rust] Dictionary encoding for strings?

Hi,

Does the Rust implementation support dictionary encoded strings?  It is not in the documentation anywhere, but there seem to be some variable-sized dictionary structs in the code base.
If not, is there a plan to support it?
Does DataFusion support reading from dictionary strings?

It seems all the examples in DataFusion and the Rust part are focused on numbers.  How robust is the string support, and how robust is the string functionality overall?

Thanks,
Evan

Re: [Rust] Dictionary encoding for strings?

Posted by Wes McKinney <we...@gmail.com>.

I believe that dictionary encoding in-memory was very recently
implemented (February 28) in
https://github.com/apache/arrow/commit/c7a7d2dcc46ed06593b994cb54c5eaf9ccd1d21d#diff-72812e30873455dcee2ce2d1ee26e4ab.
Not sure about the other questions

On Mon, Mar 9, 2020 at 11:07 PM Evan Chan <ev...@apple.com.invalid> wrote:
>
> Hi,
>
> Does the Rust implementation support dictionary encoded strings?  It is not in the documentation anywhere, but there seem to be some variable-sized dictionary structs in the code base.
> If not, is there a plan to support it?
> Does DataFusion support reading from dictionary strings?
>
> It seems all the examples in DataFusion and the Rust part are focused on numbers.  How robust is the string support, and how robust is the string functionality overall?
>
> Thanks,
> Evan