You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/10/31 19:47:23 UTC
[I] [native_datafusion] Subquery has not finished [datafusion-comet]
andygrove opened a new issue, #2672:
URL: https://github.com/apache/datafusion-comet/issues/2672
### Describe the bug
When running `CometPlanStabilitySuite` with `native_datafusion` scan, I see query failures such as:
```
- check simplified (tpcds-v1.4/q6) - native_datafusion *** FAILED *** (122 milliseconds)
java.lang.IllegalArgumentException: requirement failed: Subquery scalar-subquery#82944, [id=#198734] has not finished
at scala.Predef$.require(Predef.scala:337)
at org.apache.spark.sql.execution.ScalarSubquery.toLiteral(subquery.scala:108)
at org.apache.spark.sql.execution.FileSourceScanLike.$anonfun$pushedDownFilters$1(DataSourceScanExec.scala:426)
at org.apache.spark.sql.execution.FileSourceScanLike$$anonfun$$nestedInanonfun$translateToV1Filters$1$1.applyOrElse(DataSourceScanExec.scala:410)
at org.apache.spark.sql.execution.FileSourceScanLike$$anonfun$$nestedInanonfun$translateToV1Filters$1$1.applyOrElse(DataSourceScanExec.scala:407)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:470)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:86)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:470)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$3(TreeNode.scala:475)
at org.apache.spark.sql.catalyst.trees.BinaryLike.mapChildren(TreeNode.scala:1259)
```
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [native_datafusion] Subquery has not finished [datafusion-comet]
Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich closed issue #2672: [native_datafusion] Subquery has not finished
URL: https://github.com/apache/datafusion-comet/issues/2672
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [native_datafusion] Subquery has not finished [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #2672:
URL: https://github.com/apache/datafusion-comet/issues/2672#issuecomment-4397675106
outdated, likely fixed with recent DPP work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [native_datafusion] Subquery has not finished [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #2672: [native_datafusion] Subquery has not finished
URL: https://github.com/apache/datafusion-comet/issues/2672
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [native_datafusion] Subquery has not finished [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #2672:
URL: https://github.com/apache/datafusion-comet/issues/2672#issuecomment-3724783677
I am still seeing this issue in PR https://github.com/apache/datafusion-comet/pull/2969
```
CometTPCDSV2_7_PlanStabilitySuite:
26/01/08 09:25:14 INFO core/src/lib.rs: Comet native library version 0.13.0 initialized
- check simplified (tpcds-v2.7.0/q5a) - auto (1 second, 613 milliseconds)
- check simplified (tpcds-v2.7.0/q6) - auto *** FAILED *** (196 milliseconds)
java.lang.IllegalArgumentException: requirement failed: Subquery scalar-subquery#3400, [id=#4186] has not finished
at scala.Predef$.require(Predef.scala:337)
at org.apache.spark.sql.execution.ScalarSubquery.toLiteral(subquery.scala:108)
at org.apache.spark.sql.execution.FileSourceScanLike.$anonfun$pushedDownFilters$1(DataSourceScanExec.scala:426)
at org.apache.spark.sql.execution.FileSourceScanLike$$anonfun$$nestedInanonfun$translateToV1Filters$1$1.applyOrElse(DataSourceScanExec.scala:410)
at org.apache.spark.sql.execution.FileSourceScanLike$$anonfun$$nestedInanonfun$translateToV1Filters$1$1.applyOrElse(DataSourceScanExec.scala:407)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:470)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:86)
at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:470)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$3(TreeNode.scala:475)
at org.apache.spark.sql.catalyst.trees.BinaryLike.mapChildren(TreeNode.scala:1259)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org