You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mmaitre314 (via GitHub)" <gi...@apache.org> on 2024/02/03 23:37:07 UTC
[PR] Return an error instead of a panic when reading a corrupted Parquet file with mismatched column counts [arrow-rs]
mmaitre314 opened a new pull request, #5362:
URL: https://github.com/apache/arrow-rs/pull/5362
# Which issue does this PR close?
Closes #5315
# Rationale for this change
Data pipelines reading Parquet files may encounter corrupted Parquet files as part of their regular activities and be resilient to them through graceful handling (ex: emit telemetry and skip the files). This behavior is more natural to implement when the Parquet crate returns an error instead of a panic.
# What changes are included in this PR?
Replace an assert with an if statement.
# Are there any user-facing changes?
Caller receives an error instead of a panic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Return an error instead of a panic when reading a corrupted Parquet file with mismatched column counts [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold merged PR #5362:
URL: https://github.com/apache/arrow-rs/pull/5362
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Return an error instead of a panic when reading a corrupted Parquet file with mismatched column counts [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #5362:
URL: https://github.com/apache/arrow-rs/pull/5362#discussion_r1477158222
##########
parquet/src/file/metadata.rs:
##########
@@ -349,7 +349,13 @@ impl RowGroupMetaData {
/// Method to convert from Thrift.
pub fn from_thrift(schema_descr: SchemaDescPtr, mut rg: RowGroup) -> Result<RowGroupMetaData> {
- assert_eq!(schema_descr.num_columns(), rg.columns.len());
+ if schema_descr.num_columns() != rg.columns.len() {
+ return Err(general_err!(
+ "Column length mismatch. Schema has {} columns while Row Group has {}",
Review Comment:
```suggestion
"Column count mismatch. Schema has {} columns while Row Group has {}",
```
nit: count seems a better descriptor than length (column length made me first think of size of column, instead of number of columns)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org