You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/05/20 18:59:34 UTC

[GitHub] [arrow] nevi-me commented on a change in pull request #7220: ARROW-8837: [Rust] Implement Null data type

nevi-me commented on a change in pull request #7220:
URL: https://github.com/apache/arrow/pull/7220#discussion_r428241219



##########
File path: rust/arrow/src/ipc/writer.rs
##########
@@ -349,8 +350,9 @@ fn write_array_data(
     let null_buffer = match array_data.null_buffer() {
         None => {
             // create a buffer and fill it with valid bits
-            let buffer = MutableBuffer::new(num_rows);
-            let buffer = buffer.with_bitset(num_rows, true);
+            let num_bytes = bit_util::ceil(num_rows, 8);

Review comment:
       @andygrove @pauldix @maxburke this might be of interest for interop/integration with other implementations. I was overstating the length of the null buffer by 8. I suspect that with a sufficiently large batch size (like the 1 million that I tried with), it could cause issues with reading rust-generated IPC files.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org