You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "tustvold (via GitHub)" <gi...@apache.org> on 2023/06/05 16:38:29 UTC

[GitHub] [arrow-rs] tustvold opened a new issue, #4368: RecordReader::skip_records Is Incorrect

tustvold opened a new issue, #4368:
URL: https://github.com/apache/arrow-rs/issues/4368

   **Describe the bug**
   <!--
   A clear and concise description of what the bug is.
   -->
   
   Whilst investigating https://github.com/apache/arrow-rs/issues/4365 I realised the implementation of `RecordReader::skip_records` has two issue:
   
   * When counting buffered records to skip, it incorrectly handles the case of the final decoded record, as it doesn't know where it ends
   * When skipping buffered records, it doesn't remove the corresponding data from the decoded data, resulting in skipped rows appearing in the output
   
   **To Reproduce**
   <!--
   Steps to reproduce the behavior:
   -->
   
   https://github.com/apache/arrow-rs/issues/4365 
   
   **Expected behavior**
   <!--
   A clear and concise description of what you expected to happen.
   -->
   
   **Additional context**
   <!--
   Add any other context about the problem here.
   -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] alamb commented on issue #4368: RecordReader::skip_records Is Incorrect for Repeated Columns

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #4368:
URL: https://github.com/apache/arrow-rs/issues/4368#issuecomment-1595001497

   `label_issue.py` automatically added labels {'parquet'} from #4376


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold closed issue #4368: RecordReader::skip_records Is Incorrect for Repeated Columns

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #4368: RecordReader::skip_records Is Incorrect for Repeated Columns
URL: https://github.com/apache/arrow-rs/issues/4368


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org