You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/08/19 13:52:45 UTC

[GitHub] [arrow] alamb commented on a change in pull request #8007: ARROW-9790: [Rust][Parquet] Fix PrimitiveArrayReader boundary conditions

alamb commented on a change in pull request #8007:
URL: https://github.com/apache/arrow/pull/8007#discussion_r473045357



##########
File path: rust/parquet/src/arrow/array_reader.rs
##########
@@ -136,10 +136,8 @@ impl<T: DataType> ArrayReader for PrimitiveArrayReader<T> {
         while records_read < batch_size {
             let records_to_read = batch_size - records_read;
 
+            // NB can be 0 if at end of page
             let records_read_once = self.record_reader.read_records(records_to_read)?;
-            if records_read_once == 0 {

Review comment:
       The case of `0` rows being read is handled in the `if records_read_once < records_to-read` clause below -- namely in this case the code needs to try and get the next page of data from the page reader. 

##########
File path: rust/parquet/src/arrow/arrow_reader.rs
##########
@@ -304,6 +304,45 @@ mod tests {
         >(2, 100, 2, message_type, 15, 50, converter);
     }
 
+    #[test]

Review comment:
       Both these tests fail without the changes in this PR.
   
   I don't like the copy/paste nature of these tests and I plan a minor PR building on this one proposing how to remove the duplication and make the tests easier to read. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org