You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "comphead (via GitHub)" <gi...@apache.org> on 2025/08/28 04:43:01 UTC
[I] bug: cast StructType to String fails for duplicated col [datafusion-comet]
comphead opened a new issue, #2256:
URL: https://github.com/apache/datafusion-comet/issues/2256
### Describe the bug
The `cast StructType to StringType` fails on duplicated columns with auto and native_scan
```
checkSparkAnswerAndOperator(
"SELECT CAST(struct(_12, _12) as string) FROM tbl")
Expected only Comet native operators, but found Project.
plan: *(1) Project [cast(struct(_12, _12#11, _12, _12#11) as string) AS CAST(struct(_12, _12) AS STRING)#59]
+- *(1) CometColumnarToRow
+- CometScan [native_iceberg_compat] parquet [_12#11] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/private/var/folders/6f/_s1vnnd55zgfkx7zlwnrnv0h0000gn/T/spark-76..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_12:decimal(20,0)>
```
Found when working on #2175
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] bug: test `cast StructType to String` fails for duplicated col [datafusion-comet]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #2256:
URL: https://github.com/apache/datafusion-comet/issues/2256#issuecomment-3235337339
This happens because of
```
if (struct.names.length != struct.names.distinct.length) {
withInfo(expr, "CreateNamedStruct with duplicate field names are not supported")
return None
}
```
Having commented this out I'm getting
```
Caused by: java.lang.IllegalStateException: ArrowArray struct has 2 children (expected 1)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] feat: support `cast StructType to String` when duplicated column in the struct [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #2256:
URL: https://github.com/apache/datafusion-comet/issues/2256#issuecomment-4392215738
Closing this as not planeed for now. We have correct fallback behavior. Let's re-open if someone really needs this for performance reasons.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] bug: test `cast StructType to String` fails for duplicated col [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #2256:
URL: https://github.com/apache/datafusion-comet/issues/2256#issuecomment-3283061555
@comphead It looks like the current behavior is correct, because we fall back to Spark when there are duplicate fields? If so, can we remove the `bug` label?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] feat: support `cast StructType to String` when duplicated column in the struct [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #2256: feat: support `cast StructType to String` when duplicated column in the struct
URL: https://github.com/apache/datafusion-comet/issues/2256
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org