You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/08/17 15:22:06 UTC
[I] `cast StructType to StringType` test fails if default scan is `auto` [datafusion-comet]
andygrove opened a new issue, #2175:
URL: https://github.com/apache/datafusion-comet/issues/2175
### Describe the bug
While working on https://github.com/apache/datafusion-comet/issues/2172, I discovered a potential issue:
```
- cast StructType to StringType *** FAILED *** (428 milliseconds)
Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
Timezone Env:
== Parsed Logical Plan ==
Project [cast(struct(_9, _9#428065, _10, _10#428066, _11, _11#428067L, _12, _12#428068) as string) AS CAST(struct(_9, _10, _11, _12) AS STRING)#428125]
+- SubqueryAlias tbl
+- View (`tbl`, [_1#428057,_2#428058,_3#428059,_4#428060,_5#428061L,_6#428062,_7#428063,_8#428064,_9#428065,_10#428066,_11#428067L,_12#428068,_13#428069,_14#428070,_15#428071,_16#428072,_17#428073,_18#428074,_19#428075,_20#428076,_21#428077,_id#428078])
+- Relation [_1#428057,_2#428058,_3#428059,_4#428060,_5#428061L,_6#428062,_7#428063,_8#428064,_9#428065,_10#428066,_11#428067L,_12#428068,_13#428069,_14#428070,_15#428071,_16#428072,_17#428073,_18#428074,_19#428075,_20#428076,_21#428077,_id#428078] parquet
== Analyzed Logical Plan ==
CAST(struct(_9, _10, _11, _12) AS STRING): string
Project [cast(struct(_9, _9#428065, _10, _10#428066, _11, _11#428067L, _12, _12#428068) as string) AS CAST(struct(_9, _10, _11, _12) AS STRING)#428125]
+- SubqueryAlias tbl
+- View (`tbl`, [_1#428057,_2#428058,_3#428059,_4#428060,_5#428061L,_6#428062,_7#428063,_8#428064,_9#428065,_10#428066,_11#428067L,_12#428068,_13#428069,_14#428070,_15#428071,_16#428072,_17#428073,_18#428074,_19#428075,_20#428076,_21#428077,_id#428078])
+- Relation [_1#428057,_2#428058,_3#428059,_4#428060,_5#428061L,_6#428062,_7#428063,_8#428064,_9#428065,_10#428066,_11#428067L,_12#428068,_13#428069,_14#428070,_15#428071,_16#428072,_17#428073,_18#428074,_19#428075,_20#428076,_21#428077,_id#428078] parquet
== Optimized Logical Plan ==
Project [cast(struct(_9, _9#428065, _10, _10#428066, _11, _11#428067L, _12, _12#428068) as string) AS CAST(struct(_9, _10, _11, _12) AS STRING)#428125]
+- Relation [_1#428057,_2#428058,_3#428059,_4#428060,_5#428061L,_6#428062,_7#428063,_8#428064,_9#428065,_10#428066,_11#428067L,_12#428068,_13#428069,_14#428070,_15#428071,_16#428072,_17#428073,_18#428074,_19#428075,_20#428076,_21#428077,_id#428078] parquet
== Physical Plan ==
*(1) CometColumnarToRow
+- CometProject [CAST(struct(_9, _10, _11, _12) AS STRING)#428125], [cast(struct(_9, _9#428065, _10, _10#428066, _11, _11#428067L, _12, _12#428068) as string) AS CAST(struct(_9, _10, _11, _12) AS STRING)#428125]
+- CometScan [native_iceberg_compat] parquet [_9#428065,_10#428066,_11#428067L,_12#428068] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/spark/target/tmp/spark-f1e..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_9:smallint,_10:int,_11:bigint,_12:decimal(20,0)>
== Results ==
== Results ==
!== Correct Answer - 10000 == == Spark Answer - 10000 ==
struct<CAST(struct(_9, _10, _11, _12) AS STRING):string> struct<CAST(struct(_9, _10, _11, _12) AS STRING):string>
![{-1, -1, 4294967295, 18446744073709551615}] [{0, 0, 0, 0}]
![{-1, -1, 4294967295, 18446744073709551615}] [{0, 0, 0, 0}]
![{-1, -1, 4294967295, 18446744073709551615}] [{0, 0, 0, 0}]
![{-1, -1, 4294967295, 18446744073709551615}] [{0, 0, 0, 0}]
![{-1, -1, 4294967295, 18446744073709551615}] [{0, 0, 0, 0}]
![{-1, -1, 4294967295, 18446744073709551615}] [{0, 0, 0, 0}]
![{-1, -1, 4294967295, 18446744073709551615}] [{0, 0, 0, 0}]
![{-1, -1, 4294967295, 18446744073709551615}] [{0, 0, 0, 0}]
```
### Steps to reproduce
Update `CometTestBase` to remove the config setting the scan impl to `native_comet`
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `cast StructType to StringType` test fails if default scan is `auto` [datafusion-comet]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #2175:
URL: https://github.com/apache/datafusion-comet/issues/2175#issuecomment-3230221526
related to https://github.com/apache/datafusion-comet/pull/2246#issuecomment-3228649992
@andygrove @mbutrovich
it fails on
```
checkSparkAnswerAndOperator(
"SELECT CAST(struct(_9, _10, _11, _12) as string) FROM tbl")
```
where _9, _10 are UINTs
```
############ Column(_9) ############
name: _9
path: _9
max_definition_level: 1
max_repetition_level: 0
physical_type: INT32
logical_type: Int(bitWidth=8, isSigned=false)
converted_type (legacy): UINT_8
compression: UNCOMPRESSED (space_saved: 0%)
############ Column(_10) ############
name: _10
path: _10
max_definition_level: 1
max_repetition_level: 0
physical_type: INT32
logical_type: Int(bitWidth=16, isSigned=false)
converted_type (legacy): UINT_16
compression: UNCOMPRESSED (space_saved: 0%)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `cast StructType to StringType` test fails if default scan is `auto` [datafusion-comet]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead closed issue #2175: `cast StructType to StringType` test fails if default scan is `auto`
URL: https://github.com/apache/datafusion-comet/issues/2175
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org