[GitHub] [arrow-rs] alamb commented on a change in pull request #658: allocate enough bytes when writing booleans

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #658:
URL: https://github.com/apache/arrow-rs/pull/658#discussion_r683436247



##########
File path: parquet/src/data_type.rs
##########
@@ -674,7 +675,12 @@ pub(crate) mod private {
             bit_writer: &mut BitWriter,
         ) -> Result<()> {
             if bit_writer.bytes_written() + values.len() / 8 >= bit_writer.capacity() {
-                bit_writer.extend(256);
+                let bits_available =
+                    (bit_writer.capacity() - bit_writer.bytes_written()) * 8;
+                let bits_needed = values.len() - bits_available;
+                let bytes_needed = (bits_needed + 7) / 8;
+                let bytes_needed = round_upto_multiple_of_64(bytes_needed);
+                bit_writer.extend(bytes_needed);

Review comment:
       previously this extended by 256 bytes but now it will extend potentially by 64 byte chunks. Is that intentional? I wonder if  
   
   ```rust
                   let bytes_needed = round_upto_multiple_of_64(bytes_needed) * 4;
   ```
   
   Might be more appropriate / less of a behavior change?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org