[GitHub] [arrow] tustvold opened a new pull request #7091: ARROW-8680: [Rust] Fix ComplexObjectArray null value shifting

[GitBox <gi...@apache.org>]

tustvold opened a new pull request #7091:
URL: https://github.com/apache/arrow/pull/7091


   The null shifting logic within ComplexObjectArrayReader is incorrect as it doesn't take into account the num_readers offset within the def_levels buffer. This occurs when a read spans more than one column chunk. 
   
   Without the change the added test will fail with
   
   ```
   assertion failed: self.data.is_some()
   thread 'arrow::array_reader::tests::test_complex_array_reader_def_and_rep_levels' panicked at 'assertion failed: self.data.is_some()', parquet/src/data_type.rs:124:9
   ```
   
   This is due to the shifting logic resulting in the non-null `ByteArray` values getting shifted out of alignment with where the definition levels state that non-null values should be found.
   
   Unfortunately for the test I couldn't see an easy way to use the existing page helpers, as they assume primitive value types that can be generated randomly from a given range.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #7091: ARROW-8680: [Rust] Fix ComplexObjectArray null value shifting

[GitBox <gi...@apache.org>]

github-actions[bot] commented on pull request #7091:
URL: https://github.com/apache/arrow/pull/7091#issuecomment-623132413


   https://issues.apache.org/jira/browse/ARROW-8680


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org