You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/09/09 14:16:41 UTC
[GitHub] [arrow-datafusion] andygrove opened a new issue, #3419: DataFusion coercion logic is different from Arrow's
andygrove opened a new issue, #3419:
URL: https://github.com/apache/arrow-datafusion/issues/3419
**Describe the bug**
Arrow's `can_cast_types` has different logic than DataFusion's `coerce_types` and this results in bugs in type coercion.
**To Reproduce**
Steps to reproduce the behavior:
**Expected behavior**
A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 commented on issue #3419: DataFusion coercion logic is different from Arrow's
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on issue #3419:
URL: https://github.com/apache/arrow-datafusion/issues/3419#issuecomment-1242056342
Do we need make align the type coercion with arrow?
The cast system should not be same with arrow, because the datafusion is the SQL system and has some special semantics than the arrow lib
@andygrove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #3419: Fix inconsistencies in type coercion logic
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #3419:
URL: https://github.com/apache/arrow-datafusion/issues/3419#issuecomment-1265610564
Maybe we could return a struct like
```rust
enum NeededCoercion {
coerce_inputs: Vec<Datatype>,
output_type: DataType
}
```
Or something 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] HaoYang670 commented on issue #3419: Fix inconsistencies in type coercion logic
Posted by GitBox <gi...@apache.org>.
HaoYang670 commented on issue #3419:
URL: https://github.com/apache/arrow-datafusion/issues/3419#issuecomment-1309936300
Hi, could someone help to explain what the `coerce_types` does. Does it coerce the lhs and rhs types, or return the `return type` of the operator?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix inconsistencies in type coercion logic [datafusion]
Posted by "andygrove (via GitHub) via github" <gi...@datafusion.apache.org>.
andygrove closed issue #3419: Fix inconsistencies in type coercion logic
URL: https://github.com/apache/datafusion/issues/3419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
[GitHub] [arrow-datafusion] andygrove commented on issue #3419: DataFusion coercion logic is different from Arrow's
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #3419:
URL: https://github.com/apache/arrow-datafusion/issues/3419#issuecomment-1242091025
I agree that there will be differences. See the conversation at https://github.com/apache/arrow-datafusion/pull/3407#discussion_r967137788 for one example where we have to work around a difference and this example is possibly due to a bug in DataFusions logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on issue #3419: Fix inconsistencies in type coercion logic
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #3419:
URL: https://github.com/apache/arrow-datafusion/issues/3419#issuecomment-1242181541
I updated the title and description now that I understand this more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #3419: Fix inconsistencies in type coercion logic
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #3419:
URL: https://github.com/apache/arrow-datafusion/issues/3419#issuecomment-1310185079
> Hi, could someone help to explain what the coerce_types does. Does it coerce the lhs and rhs types, or calculate the return type of the operator?
I think this is a reasonable description of coercion:
https://github.com/apache/arrow-datafusion/blob/9c24a79118c0ae0aee480bf039385a06d240b499/datafusion/expr/src/type_coercion.rs#L18-L32
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Fix inconsistencies in type coercion logic [datafusion]
Posted by "andygrove (via GitHub) via github" <gi...@datafusion.apache.org>.
andygrove commented on issue #3419:
URL: https://github.com/apache/datafusion/issues/3419#issuecomment-6933007396
Closing as fixed, by #6778 and later refactors. Binary coercion now goes through `BinaryTypeCoercer`, which keeps `get_input_types()` (the types to cast the left and right sides to) separate from `get_result_type()` (the output type of the operator). That separation is what this issue asked for.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org