You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/27 18:45:45 UTC
[GitHub] [arrow-rs] tustvold opened a new pull request, #2201: Add filter pushdown example (#1191)
tustvold opened a new pull request, #2201:
URL: https://github.com/apache/arrow-rs/pull/2201
Adds an example of filter evaluation as described in #1191
Currently results in `ParquetError("Parquet error: failed to skip rows, expected 12, got 7")`. I suspect something isn't quite right yet, FYI @Ted-Jiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #2201: Add filter pushdown example (#1191)
Posted by GitBox <gi...@apache.org>.
tustvold commented on PR #2201:
URL: https://github.com/apache/arrow-rs/pull/2201#issuecomment-1197295517
Rebasing on top of #2199 appears to fix the "failed to skip rows" issue, now just to work out why the data is incorrect :sweat_smile:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #2201: Add filter pushdown example (#1191)
Posted by GitBox <gi...@apache.org>.
tustvold commented on PR #2201:
URL: https://github.com/apache/arrow-rs/pull/2201#issuecomment-1206528721
Closing in favor or #2335
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on a diff in pull request #2201: Add filter pushdown example (#1191)
Posted by GitBox <gi...@apache.org>.
tustvold commented on code in PR #2201:
URL: https://github.com/apache/arrow-rs/pull/2201#discussion_r931474006
##########
parquet/src/file/page_index/index_reader.rs:
##########
@@ -35,7 +35,7 @@ pub fn read_columns_indexes<R: ChunkReader>(
let length = lengths.iter().sum::<usize>();
//read all need data into buffer
- let mut reader = reader.get_read(offset, reader.len() as usize)?;
+ let mut reader = reader.get_read(offset, length)?;
Review Comment:
These were actually bugs that were only working because the ChunkReader implementation for File doesn't validate the end offset is actually within the range of the file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold closed pull request #2201: Add filter pushdown example (#1191)
Posted by GitBox <gi...@apache.org>.
tustvold closed pull request #2201: Add filter pushdown example (#1191)
URL: https://github.com/apache/arrow-rs/pull/2201
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org