You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "JkSelf (via GitHub)" <gi...@apache.org> on 2025/07/08 01:27:03 UTC
[PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
JkSelf opened a new pull request, #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
(Fixes: \#ISSUE-ID)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3051004328
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "JkSelf (via GitHub)" <gi...@apache.org>.
JkSelf commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2206387246
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -63,13 +65,26 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
* Validate whether this SparkPlan supports to be transformed into substrait node in Native Code.
*/
final def doValidate(): ValidationResult = {
- val schemaValidationResult = BackendsApiManager.getValidatorApiInstance
- .doSchemaValidate(schema)
- .map {
- reason =>
- ValidationResult.failed(s"Found schema check failure for $schema, due to: $reason")
+ val schemaValidationResult =
+ try {
+ BackendsApiManager.getValidatorApiInstance
+ .doSchemaValidate(schema)
+ .map {
+ reason =>
+ ValidationResult.failed(
+ s"Found schema check failure for $schema, due to: $reason")
+ }
+ .getOrElse(ValidationResult.succeeded)
+ } catch {
+ case e: AnalysisException =>
Review Comment:
@PHILO-HE Yes. Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "JkSelf (via GitHub)" <gi...@apache.org>.
JkSelf commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2194406833
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -43,7 +43,7 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
try {
f
} catch {
- case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException) =>
+ case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException | _: Throwable) =>
Review Comment:
@zhztheplayer @PHILO-HE Thanks for your detailed explanation. @FelixYBW Updated to only catch the `AnalysisException`. Please help to review again. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3051002352
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2193958230
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -43,7 +43,7 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
try {
f
} catch {
- case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException) =>
+ case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException | _: Throwable) =>
Review Comment:
Would prefer catching `Exception` so we still allow fatal errors (e.g., a heap OOM) to crash the application.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3050991302
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3051679882
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3071766862
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2206217834
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -63,13 +65,26 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
* Validate whether this SparkPlan supports to be transformed into substrait node in Native Code.
*/
final def doValidate(): ValidationResult = {
- val schemaValidationResult = BackendsApiManager.getValidatorApiInstance
- .doSchemaValidate(schema)
- .map {
- reason =>
- ValidationResult.failed(s"Found schema check failure for $schema, due to: $reason")
+ val schemaValidationResult =
+ try {
+ BackendsApiManager.getValidatorApiInstance
+ .doSchemaValidate(schema)
+ .map {
+ reason =>
+ ValidationResult.failed(
+ s"Found schema check failure for $schema, due to: $reason")
+ }
+ .getOrElse(ValidationResult.succeeded)
+ } catch {
+ case e: AnalysisException =>
Review Comment:
Suggest to use `UnsolvedException`, 1) it's more specific 2) To make this exception catching also works Spark 4.0.
Prior to Spark 4.0, UnsolvedException extends AnalysisException, but since Spark 4.0, it extends SparkException.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103][VL] Fall back to vanilla spark when UnresolvedException occurs in the schema validation [incubator-gluten]
Posted by "JkSelf (via GitHub)" <gi...@apache.org>.
JkSelf merged PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "JkSelf (via GitHub)" <gi...@apache.org>.
JkSelf commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3055026810
@zhztheplayer @PHILO-HE @FelixYBW Do you have any further comment? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3047050416
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "JkSelf (via GitHub)" <gi...@apache.org>.
JkSelf commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2194404141
##########
docs/velox-backend-limitations.md:
##########
@@ -163,3 +163,6 @@ Gluten's.
### CSV Read
The header option should be true. And now we only support DatasourceV1, i.e., user should set `spark.sql.sources.useV1SourceList=csv`. User defined read option is not supported, which will make CSV read fall back to vanilla Spark in most case.
CSV read will also fall back to vanilla Spark and log warning when user specifies schema is different with file schema.
+
+### Utilizing Map Type as Hash Keys in ColumnarShuffleExchange
+Spark uses the `spark.sql.legacy.allowHashOnMapType` configuration to support hash map key functions. Gluten enables this configuration during the creation of ColumnarShuffleExchange, as shown in the code [link](https://github.com/apache/incubator-gluten/blob/main/backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxSparkPlanExecApi.scala#L355-L363). This method bypasses Spark's unresolved checks and creates projects with the hash(mapType) operator before ColumnarShuffleExchange. However, if `spark.sql.legacy.allowHashOnMapType` is disabled in a test environment, projects using the hash(mapType) expression may throw an `Invalid call to dataType on unresolved object` exception during validation, causing them to fallback to vanilla Spark, as referenced in the code [link](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala#L291-L296). Enabling this configuration allows the project to be offloaded
to Velox.
Review Comment:
@PHILO-HE Yes. Updated. Please help to review again. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2194230599
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -43,7 +43,7 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
try {
f
} catch {
- case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException) =>
+ case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException | _: Throwable) =>
Review Comment:
can we catch the exact exception instead of all Throwable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3047049826
https://github.com/apache/incubator-gluten/issues/10103
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3050988546
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2194059718
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -43,7 +43,7 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
try {
f
} catch {
- case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException) =>
+ case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException | _: Throwable) =>
Review Comment:
Generally JVM heap OGenerally JVM heap OOM could be raised at any point of the Java / Scala code. It's just an example of fatal errors. There are more of them like illegal access errors, class compatibility errors, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2194068798
##########
docs/velox-backend-limitations.md:
##########
@@ -163,3 +163,6 @@ Gluten's.
### CSV Read
The header option should be true. And now we only support DatasourceV1, i.e., user should set `spark.sql.sources.useV1SourceList=csv`. User defined read option is not supported, which will make CSV read fall back to vanilla Spark in most case.
CSV read will also fall back to vanilla Spark and log warning when user specifies schema is different with file schema.
+
+### Utilizing Map Type as Hash Keys in ColumnarShuffleExchange
+Spark uses the `spark.sql.legacy.allowHashOnMapType` configuration to support hash map key functions. Gluten enables this configuration during the creation of ColumnarShuffleExchange, as shown in the code [link](https://github.com/apache/incubator-gluten/blob/main/backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxSparkPlanExecApi.scala#L355-L363). This method bypasses Spark's unresolved checks and creates projects with the hash(mapType) operator before ColumnarShuffleExchange. However, if `spark.sql.legacy.allowHashOnMapType` is disabled in a test environment, projects using the hash(mapType) expression may throw an `Invalid call to dataType on unresolved object` exception during validation, causing them to fallback to vanilla Spark, as referenced in the code [link](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala#L291-L296). Enabling this configuration allows the project to be offloaded
to Velox.
Review Comment:
Break this line into multiple code lines.
Please use github permanent link to avoid unexpected reference once code line is changed.
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -43,7 +43,7 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
try {
f
} catch {
- case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException) =>
+ case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException | _: Throwable) =>
Review Comment:
Similarly, I feel it may not be good to catch all exceptions which can hide some critical issues behind the fallback behavior.
Can we detect this exception somehow and throw GlutenNotSupportException instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3051656256
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "JkSelf (via GitHub)" <gi...@apache.org>.
JkSelf commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2194005610
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -43,7 +43,7 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
try {
f
} catch {
- case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException) =>
+ case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException | _: Throwable) =>
Review Comment:
Does the OOM issue arise during the validation stage?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs exception [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on code in PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#discussion_r2194059718
##########
gluten-substrait/src/main/scala/org/apache/gluten/execution/ValidatablePlan.scala:
##########
@@ -43,7 +43,7 @@ trait ValidatablePlan extends GlutenPlan with LogLevelUtil {
try {
f
} catch {
- case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException) =>
+ case e @ (_: GlutenNotSupportException | _: UnsupportedOperationException | _: Throwable) =>
Review Comment:
Generally JVM heap OOM could be raised at any point of the Java / Scala code. It's just an example of fatal errors. There are more of them like illegal access errors, class compatibility errors, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-10103] [VL] Fallback to vanilla spark when the validation stage occurs AnalysisException [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10138:
URL: https://github.com/apache/incubator-gluten/pull/10138#issuecomment-3071940069
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org