You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "Surbhi-Vijay (via GitHub)" <gi...@apache.org> on 2026/01/09 11:02:16 UTC
[PR] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Surbhi-Vijay opened a new pull request, #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387
## What changes are proposed in this pull request?
Schema validation fails during validation phase for unsupported join types. When an unsupported join type is encountered, it throws `IllegalArgumentException `. This patch catches the exception and adds validation failure msg. This helps in falling back unsupported join type to Spark and query successfully completes.
This issue was encountered with Spark-4.0 where new join type `LeftSingle` has been introduced.
See: https://github.com/apache/incubator-gluten/issues/11372
## How was this patch tested?
Tested locally by running spark testcase with native session enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#issuecomment-3737306029
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#issuecomment-3737354345
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "Surbhi-Vijay (via GitHub)" <gi...@apache.org>.
Surbhi-Vijay commented on code in PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#discussion_r2681162748
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -85,6 +85,12 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
s" consider enabling the spark.sql.legacy.allowHashOnMapType " +
s"setting to resolve this issue."
ValidationResult.failed(message)
+ case e: IllegalArgumentException =>
+ // SchemaValidation throws IllegalArgumentException in Join validation
+ // when join type is unsupported. For example,
+ // LeftSingle join in BroadcastHashJoinExecTransformer.
+ ValidationResult.failed(
+ s"Failed to retrieve schema for ${this.nodeName}, due to: ${e.getMessage}")
}
Review Comment:
Do you mean matching the exception message or changing the exception itself?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [BUG-FIX] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "Surbhi-Vijay (via GitHub)" <gi...@apache.org>.
Surbhi-Vijay commented on PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#issuecomment-3728924471
<img width="1170" height="768" alt="image" src="https://github.com/user-attachments/assets/911da8f8-7210-4d9f-8c1a-a35c5fe01222" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "liuneng1994 (via GitHub)" <gi...@apache.org>.
liuneng1994 merged PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#issuecomment-3741964954
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "Surbhi-Vijay (via GitHub)" <gi...@apache.org>.
Surbhi-Vijay commented on code in PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#discussion_r2681162748
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -85,6 +85,12 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
s" consider enabling the spark.sql.legacy.allowHashOnMapType " +
s"setting to resolve this issue."
ValidationResult.failed(message)
+ case e: IllegalArgumentException =>
+ // SchemaValidation throws IllegalArgumentException in Join validation
+ // when join type is unsupported. For example,
+ // LeftSingle join in BroadcastHashJoinExecTransformer.
+ ValidationResult.failed(
+ s"Failed to retrieve schema for ${this.nodeName}, due to: ${e.getMessage}")
}
Review Comment:
Do you mean the matching exception message or changing the exception itself?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "Surbhi-Vijay (via GitHub)" <gi...@apache.org>.
Surbhi-Vijay commented on code in PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#discussion_r2681245488
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -85,6 +85,12 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
s" consider enabling the spark.sql.legacy.allowHashOnMapType " +
s"setting to resolve this issue."
ValidationResult.failed(message)
+ case e: IllegalArgumentException =>
+ // SchemaValidation throws IllegalArgumentException in Join validation
+ // when join type is unsupported. For example,
+ // LeftSingle join in BroadcastHashJoinExecTransformer.
+ ValidationResult.failed(
+ s"Failed to retrieve schema for ${this.nodeName}, due to: ${e.getMessage}")
}
Review Comment:
I think it should be okay to check `IllegalArgumentException`. We are catching this exception during validation phase and even if the validation is failing at some other place than joins, it should fallback to Spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#issuecomment-3737710521
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#issuecomment-3728440536
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [BUG-FIX] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "Yohahaha (via GitHub)" <gi...@apache.org>.
Yohahaha commented on code in PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#discussion_r2680696500
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -85,6 +85,12 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
s" consider enabling the spark.sql.legacy.allowHashOnMapType " +
s"setting to resolve this issue."
ValidationResult.failed(message)
+ case e: IllegalArgumentException =>
+ // SchemaValidation throws IllegalArgumentException in Join validation
+ // when join type is unsupported. For example,
+ // LeftSingle join in BroadcastHashJoinExecTransformer.
+ ValidationResult.failed(
+ s"Failed to retrieve schema for ${this.nodeName}, due to: ${e.getMessage}")
}
Review Comment:
Should we match Exception to avoid more special case? maybe IllegalArgumentException will be throw in somewhere else.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [CORE][VL] Handle exception in Schema validation for unsupported join types [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11387:
URL: https://github.com/apache/incubator-gluten/pull/11387#issuecomment-3747724873
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org