You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "felipepessoto (via GitHub)" <gi...@apache.org> on 2026/05/12 00:52:33 UTC
[PR] [VL] Local files node variant fallback fix [gluten]
felipepessoto opened a new pull request, #12074:
URL: https://github.com/apache/gluten/pull/12074
## What changes are proposed in this pull request?
Reading the Change Data Feed (CDF) of a Delta table that contains a VARIANT column
fails under the Velox native engine when the variant column is **not** projected:
```
spark.sql("""CREATE TABLE tbl (id INT, data VARIANT) USING delta
TBLPROPERTIES ('delta.enableChangeDataFeed' = 'true')""")
spark.sql("""INSERT INTO tbl
SELECT 1, parse_json('{"status":"active"}') UNION ALL
SELECT 2, parse_json('{"status":"active"}')""")
// Works (variant projected → existing variant validation falls back to vanilla Spark)
spark.read.format("delta").option("readChangeFeed", "true").option("startingVersion", 0)
.table("tbl").select("id", "data", "_change_type").show()
// Fails with GlutenNotSupportException: Type VariantType not supported
spark.read.format("delta").option("readChangeFeed", "true").option("startingVersion", 0)
.table("tbl").select("id", "_change_type").show()
```
```
org.apache.gluten.exception.GlutenNotSupportException: Type VariantType not supported.
at org.apache.gluten.expression.ConverterUtils$.getTypeNode(ConverterUtils.scala:248)
at org.apache.gluten.substrait.rel.LocalFilesNode.buildNamedStruct(LocalFilesNode.java:115)
at org.apache.gluten.substrait.rel.LocalFilesNode.toProtobuf(LocalFilesNode.java:199)
at org.apache.gluten.backendsapi.velox.VeloxIteratorApi
.$anonfun$genFirstStageIterator$3(VeloxIteratorApi.scala:224)
...
```
## How was this patch tested?
New unit tests. Also tested with Delta unit tests.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Github Copilot CLI v1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] LocalFilesNode Variant Type fallback fix [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto commented on PR #12074:
URL: https://github.com/apache/gluten/pull/12074#issuecomment-4492073792
@baibaichen could you help review this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] Local files node variant fallback fix [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12074:
URL: https://github.com/apache/gluten/pull/12074#issuecomment-4426349313
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] Local files node variant fallback fix [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12074:
URL: https://github.com/apache/gluten/pull/12074#issuecomment-4455285913
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] Local files node variant fallback fix [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12074:
URL: https://github.com/apache/gluten/pull/12074#issuecomment-4455320596
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] LocalFilesNode Variant Type fallback fix [gluten]
Posted by "felipepessoto (via GitHub)" <gi...@apache.org>.
felipepessoto closed pull request #12074: [VL] LocalFilesNode Variant Type fallback fix
URL: https://github.com/apache/gluten/pull/12074
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org