You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/29 07:28:52 UTC
[GitHub] [arrow-datafusion] silence-coding opened a new pull request #2113: issue#1967 ignore channel close
silence-coding opened a new pull request #2113:
URL: https://github.com/apache/arrow-datafusion/pull/2113
#1967 The sending failure occurs only when the receiving end closes the channel. In this case, I do not think we need to print this error log, especially when the limit statement is used normally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] silence-coding commented on a change in pull request #2113: issue#1967 ignore channel close
Posted by GitBox <gi...@apache.org>.
silence-coding commented on a change in pull request #2113:
URL: https://github.com/apache/arrow-datafusion/pull/2113#discussion_r837144987
##########
File path: datafusion/core/src/physical_plan/file_format/parquet.rs
##########
@@ -515,10 +516,10 @@ fn read_partition(
let err_msg =
format!("Error reading batch from {}: {}", partitioned_file, e);
// send error to operator
- send_result(
+ let _ = send_result(
&response_tx,
Err(ArrowError::ParquetError(err_msg.clone())),
- )?;
Review comment:
Here the send_result error should not overwrite the original ArrowError, so I ignored it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] silence-coding commented on pull request #2113: issue#1967 ignore channel close
Posted by GitBox <gi...@apache.org>.
silence-coding commented on pull request #2113:
URL: https://github.com/apache/arrow-datafusion/pull/2113#issuecomment-1081518781
@alamb Please check it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a change in pull request #2113: issue#1967 ignore channel close
Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #2113:
URL: https://github.com/apache/arrow-datafusion/pull/2113#discussion_r837299762
##########
File path: datafusion/core/src/physical_plan/file_format/parquet.rs
##########
@@ -503,8 +503,9 @@ fn read_partition(
let proj_batch = partition_column_projector
.project(adapted_batch, &partitioned_file.partition_values);
- send_result(&response_tx, proj_batch)?;
- if limit.map(|l| total_rows >= l).unwrap_or(false) {
+ let send_rt = send_result(&response_tx, proj_batch);
Review comment:
I think it makes sense not to error if `send_result` fails due to the receiver being closed. I checked that this is the only error `send_result` can make: https://github.com/apache/arrow-datafusion/blob/41b4e491663029f653e491b110d0b5e74d08a0b6/datafusion/core/src/physical_plan/file_format/parquet.rs#L309
👍
##########
File path: datafusion/core/src/physical_plan/file_format/parquet.rs
##########
@@ -515,10 +516,10 @@ fn read_partition(
let err_msg =
format!("Error reading batch from {}: {}", partitioned_file, e);
// send error to operator
- send_result(
+ let _ = send_result(
Review comment:
```suggestion
// send_result error, if any, should not overwrite the original ArrowError, so ignore it
let _ = send_result(
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #2113: issue#1967 ignore channel close
Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #2113:
URL: https://github.com/apache/arrow-datafusion/pull/2113#issuecomment-1083409888
Thanks agian @silence-coding
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] silence-coding commented on a change in pull request #2113: issue#1967 ignore channel close
Posted by GitBox <gi...@apache.org>.
silence-coding commented on a change in pull request #2113:
URL: https://github.com/apache/arrow-datafusion/pull/2113#discussion_r837146307
##########
File path: datafusion/core/src/physical_plan/file_format/parquet.rs
##########
@@ -503,8 +503,9 @@ fn read_partition(
let proj_batch = partition_column_projector
.project(adapted_batch, &partitioned_file.partition_values);
- send_result(&response_tx, proj_batch)?;
- if limit.map(|l| total_rows >= l).unwrap_or(false) {
+ let send_rt = send_result(&response_tx, proj_batch);
Review comment:
When both the where statement and the limit statement are used, the value of the limit is None. As a result, the normal limit statement also triggers the channel close error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a change in pull request #2113: issue#1967 ignore channel close
Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #2113:
URL: https://github.com/apache/arrow-datafusion/pull/2113#discussion_r838788639
##########
File path: datafusion/core/src/physical_plan/file_format/parquet.rs
##########
@@ -515,10 +516,10 @@ fn read_partition(
let err_msg =
format!("Error reading batch from {}: {}", partitioned_file, e);
// send error to operator
- send_result(
+ let _ = send_result(
Review comment:
https://github.com/apache/arrow-datafusion/pull/2127
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb merged pull request #2113: issue#1967 ignore channel close
Posted by GitBox <gi...@apache.org>.
alamb merged pull request #2113:
URL: https://github.com/apache/arrow-datafusion/pull/2113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org