[Rust] Vectorized traits for using arrays outside Arrow

[Vertexclique <ve...@gmail.com>]

Hi;

I wanted to discuss with Rust lib maintainers about how can we improve the current status of Rust's DictionaryArray and reading its' encoding array outside the Arrow. So a simple predicate filter needs to collect index over iterator and flat map over the optional values or map over the None values and replace with sentinel values. Thou iterator is written nearly frictionless and overheadless by the implementor. (Congrats it looks nice!) Still there is an overhead of iterator and yielding of elements inside iterator implementation.

So I propose a simple trait called "Vectorized" which will allow us to dispense arrays with defined type with requested sentinels. This approach will work zerocopy and will use underlying Buffer type.

I am eagerly waiting for your input and I would like to clarify more if needed.

Best,
Mahmut