You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "yukkit (via GitHub)" <gi...@apache.org> on 2023/11/10 08:46:13 UTC
[I] Function 'struct' may change the data type of input parameters [arrow-datafusion]
yukkit opened a new issue, #8118:
URL: https://github.com/apache/arrow-datafusion/issues/8118
When I construct the structure through the function `struct`, my original data type is changed. I don't know if this is a bug or an expected design, but I think a better way is to retain the original type
**To Reproduce**
```
CREATE TABLE values(
c0 INT,
c1 String,
c2 String
) AS VALUES
(1, 'a', 'a'),
(2, 'b', 'b'),
(3, 'c', 'c');
explain verbose select struct(c0, c1, c2) from VALUES;
```
```
+------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+
| plan_type | plan |
+------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+
| initial_logical_plan | Projection: struct(values.c0, values.c1, values.c2) |
| | TableScan: values |
| logical_plan after inline_table_scan | SAME TEXT AS ABOVE |
| logical_plan after type_coercion | Projection: struct(CAST(values.c0 AS Utf8), values.c1, values.c2) |
| | TableScan: values |
| logical_plan after count_wildcard_rule | SAME TEXT AS ABOVE |
| analyzed_logical_plan | SAME TEXT AS ABOVE
```
I see AnalyzerRule `type_coercion` cast `values.c0` to `values.c0 AS Utf8`.
I also have some questions, why struct only supports the following types? @Ted-Jiang My idea is that struct can support any type. Is it possible to use TypeSignature::VariadicAny type of function signature?
```
/// Currently supported types by the struct function.
pub static SUPPORTED_STRUCT_TYPES: &[DataType] = &[
DataType::Boolean,
DataType::UInt8,
DataType::UInt16,
DataType::UInt32,
DataType::UInt64,
DataType::Int8,
DataType::Int16,
DataType::Int32,
DataType::Int64,
DataType::Float32,
DataType::Float64,
DataType::Utf8,
DataType::LargeUtf8,
];
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Function 'struct' may change the data type of input parameters [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #8118: Function 'struct' may change the data type of input parameters
URL: https://github.com/apache/arrow-datafusion/issues/8118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Function 'struct' may change the data type of input parameters [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #8118:
URL: https://github.com/apache/arrow-datafusion/issues/8118#issuecomment-1808747260
I also looked at the implementation in
https://github.com/apache/arrow-datafusion/blob/3df895597d8c2073081fd9d990048c7aefb3b62e/datafusion/physical-expr/src/struct_expressions.rs#L32-L63
I agree there is no reason to restrict the types supported as field types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org