[I] `StructArray::try_new_with_length` can panic given null array with max length [arrow-rs]

["Jefffrey (via GitHub)" <gi...@apache.org>]

Jefffrey opened a new issue, #11848:
URL: https://github.com/apache/arrow-rs/issues/11848

   ### Describe the bug
   
   Given a null array with length `usize::MAX`, and if we try to create a struct array containing this array but with corresponding field as non-nullable, `StructArray::try_new_with_length` can panic instead of erroring.
   
   https://github.com/apache/arrow-rs/blob/d03274a7f69d9ec18e2d74920f8b880dd0a722a0/arrow-array/src/array/struct_array.rs#L169-L178
   
   Specifically here, when calling `logical_nulls()` it'll materialize an entire null buffer with `usize::MAX` bits
   
   https://github.com/apache/arrow-rs/blob/d03274a7f69d9ec18e2d74920f8b880dd0a722a0/arrow-array/src/array/null_array.rs#L120-L122
   
   Which leads to a memory allocation error
   
   ### To Reproduce
   
   ```rust
       #[test]
       fn test123() {
           let fields = Fields::from(vec![Field::new("a", DataType::Null, false)]);
           let array = Arc::new(NullArray::new(usize::MAX));
           let a = StructArray::try_new_with_length(fields, vec![array], None, usize::MAX);
           assert!(a.is_err());
       }
   ```
   
   Running this test panics:
   
   ```sh
   running 1 test
   test array::struct_array::tests::test123 ... FAILED
   
   failures:
   
   ---- array::struct_array::tests::test123 stdout ----
   
   thread 'array::struct_array::tests::test123' (1106047) panicked at arrow-buffer/src/buffer/mutable.rs:202:59:
   failed to allocate memory for layout Layout { size: 2305843009213693952, align: 64 (1 << 6) }
   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
   
   
   failures:
       array::struct_array::tests::test123
   
   test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 747 filtered out; finished in 0.00s
   
   error: test failed, to rerun pass `-p arrow-array --lib`
   ```
   
   ### Expected behavior
   
   This should return an error, and not panic
   
   ### Additional context
   
   Reported by Adam Korczynski from ADA Logics, identified by Anthropic


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] `StructArray::try_new_with_length` can panic given null array with max length [arrow-rs]

["llxgdtop (via GitHub) via github" <gi...@arrow.apache.org>]

llxgdtop commented on issue #11848:
URL: https://github.com/apache/arrow-rs/issues/11848#issuecomment-6658792645

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org