You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "etseidl (via GitHub)" <gi...@apache.org> on 2025/11/04 00:44:00 UTC
[PR] [Parquet] Allow reading of files with unknown logical types [arrow-rs]
etseidl opened a new pull request, #8777:
URL: https://github.com/apache/arrow-rs/pull/8777
# Which issue does this PR close?
- Closes #8776.
# Rationale for this change
See issue
# What changes are included in this PR?
Modifies a few conversion functions to account for unknown logical types
# Are these changes tested?
Yes, tests are added
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Parquet] Allow reading of files with unknown logical types [arrow-rs]
Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl commented on PR #8777:
URL: https://github.com/apache/arrow-rs/pull/8777#issuecomment-3491702729
Thanks @mbrobbel!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Parquet] Allow reading of files with unknown logical types [arrow-rs]
Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl merged PR #8777:
URL: https://github.com/apache/arrow-rs/pull/8777
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Parquet] Allow reading of files with unknown logical types [arrow-rs]
Posted by "etseidl (via GitHub)" <gi...@apache.org>.
etseidl commented on code in PR #8777:
URL: https://github.com/apache/arrow-rs/pull/8777#discussion_r2490613806
##########
parquet/src/file/serialized_reader.rs:
##########
@@ -2696,4 +2696,23 @@ mod tests {
);
}
}
+
+ #[test]
+ fn test_read_unknown_logical_type() {
+ let file = get_test_file("unknown-logical-type.parquet");
+ let reader = SerializedFileReader::new(file).unwrap();
+ let mut iter = reader
+ .get_row_iter(None)
+ .expect("Failed to create row iterator");
+
+ let mut start = 0;
+ let end = reader.metadata().file_metadata().num_rows();
+ while start < end {
+ match iter.next() {
+ Some(_) => {}
+ None => break,
+ };
+ start += 1;
+ }
Review Comment:
Sorry, I was a little rushed. The real test is that you can create the reader. I'll clean this up a bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Parquet] Allow reading of files with unknown logical types [arrow-rs]
Posted by "mbrobbel (via GitHub)" <gi...@apache.org>.
mbrobbel commented on code in PR #8777:
URL: https://github.com/apache/arrow-rs/pull/8777#discussion_r2489158101
##########
parquet/src/file/serialized_reader.rs:
##########
@@ -2696,4 +2696,23 @@ mod tests {
);
}
}
+
+ #[test]
+ fn test_read_unknown_logical_type() {
+ let file = get_test_file("unknown-logical-type.parquet");
+ let reader = SerializedFileReader::new(file).unwrap();
+ let mut iter = reader
+ .get_row_iter(None)
+ .expect("Failed to create row iterator");
+
+ let mut start = 0;
+ let end = reader.metadata().file_metadata().num_rows();
+ while start < end {
+ match iter.next() {
+ Some(_) => {}
+ None => break,
+ };
+ start += 1;
+ }
Review Comment:
Should we `assert_eq!(start, end)` here or is it good enough if this doesn't panic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org