[GitHub] [arrow-rs] tmcw commented on issue #4804: Different encoding options appear to have no effect on output size

["tmcw (via GitHub)" <gi...@apache.org>]

tmcw commented on issue #4804:
URL: https://github.com/apache/arrow-rs/issues/4804#issuecomment-1712588472

   Got it, thanks! Disabling dictionary encoding with
   
   ```rs
       let props = WriterProperties::builder()
           .set_dictionary_enabled(false)
   ```
   
   Made changing the encoding with `set_encoding` actually have an effect, and ends up that dictionary encoding seems like the only beneficial setting for this data - plain is larger, delta binary packed is also a little larger, and the rest either fail because they're not compatible with the data type or they're the dictionary encoding.
   
   It's a little confusing for newcomers, I think, the relationship between `set_encoding` and `set_dictionary_enabled` - I guess `set_encoding` is a no-op if dictionary is enabled? Happy to contribute a docs note if that's the case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org