[GitHub] [arrow-rs] bjchambers commented on a change in pull request #658: allocate enough bytes when writing booleans

[GitBox <gi...@apache.org>]

bjchambers commented on a change in pull request #658:
URL: https://github.com/apache/arrow-rs/pull/658#discussion_r683595268



##########
File path: parquet/src/data_type.rs
##########
@@ -674,7 +675,12 @@ pub(crate) mod private {
             bit_writer: &mut BitWriter,
         ) -> Result<()> {
             if bit_writer.bytes_written() + values.len() / 8 >= bit_writer.capacity() {
-                bit_writer.extend(256);
+                let bits_available =
+                    (bit_writer.capacity() - bit_writer.bytes_written()) * 8;
+                let bits_needed = values.len() - bits_available;
+                let bytes_needed = (bits_needed + 7) / 8;
+                let bytes_needed = round_upto_multiple_of_64(bytes_needed);
+                bit_writer.extend(bytes_needed);

Review comment:
       Totally unintentional -- I wasn't sure what the desired growth pattern was beyond "it should grow by at least enough to hold everything". I'm happy to make a change like that, although it may be clearer to do `round_upto_power_of_2(bytes_needed, 256)`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org