You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/02/15 21:04:54 UTC
[GitHub] [arrow-datafusion] alamb opened a new pull request #1837: Return `Error` when parquet reader fails rather than no data with `println!`
alamb opened a new pull request #1837:
URL: https://github.com/apache/arrow-datafusion/pull/1837
# Which issue does this PR close?
Fixes: https://github.com/apache/arrow-datafusion/issues/1767 (cc @andygrove )
I believe this also fixes https://github.com/apache/arrow-datafusion/issues/1651 (cc @thinkharderdev )
# Rationale for this change
Errors are getting ignored during processing. See #1767 for details
# What changes are included in this PR?
1. Send the error back to the `ParquetExec` stream
2. Tests for same
# Are there any user-facing changes?
Errors are reported when there are real errors
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Dandandan merged pull request #1837: Return `Error` when parquet reader fails rather than no data with `println!`
Posted by GitBox <gi...@apache.org>.
Dandandan merged pull request #1837:
URL: https://github.com/apache/arrow-datafusion/pull/1837
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a change in pull request #1837: Return `Error` when parquet reader fails rather than no data with `println!`
Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #1837:
URL: https://github.com/apache/arrow-datafusion/pull/1837#discussion_r807328426
##########
File path: datafusion/src/physical_plan/file_format/parquet.rs
##########
@@ -808,18 +822,8 @@ mod tests {
let read =
round_trip_to_parquet(vec![batch1, batch2], None, Some(Arc::new(schema)))
.await;
-
- // expect only the first batch to be read
- let expected = vec![
- "+-----+----+----+",
- "| c1 | c2 | c3 |",
- "+-----+----+----+",
- "| Foo | 1 | 10 |",
- "| | 2 | 20 |",
- "| bar | | |",
- "+-----+----+----+",
- ];
- assert_batches_sorted_eq!(expected, &read);
+ assert_contains!(read.unwrap_err().to_string(),
Review comment:
This test should have always failed -- see https://github.com/apache/arrow-datafusion/pull/1622#discussion_r790137450
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org