You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2023/12/02 11:42:23 UTC
Re: [PR] [SPARK-45943][SQL] Move DetermineTableStats to resolution rules [spark]
LuciferYang commented on PR #43867:
URL: https://github.com/apache/spark/pull/43867#issuecomment-1837128189
After merging this pr, there are test failures in branch-3.5. Can you take a look at this issue? @wForget
https://github.com/apache/spark/actions/runs/7041075997/job/19163039661
```
[info] HiveSourceRowLevelOperationSuite:
03:30:31.518 WARN org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog: A Hive serde table will be created as there is no table provider specified. You can set spark.sql.legacy.createHiveTableByDefault to false so that native data source table will be created instead.
03:30:31.535 WARN org.apache.hadoop.hive.metastore.HiveMetaStore: Location: file:/home/runner/work/spark/spark/target/tmp/warehouse-b721773d-f779-4db4-9ebc-ca8315b15f61/hive_table specified for non-external table:hive_table
[info] - SPARK-45943: merge into using hive table without stats *** FAILED *** (391 milliseconds)
[info] org.apache.spark.sql.AnalysisException: LEGACY store assignment policy is disallowed in Spark data source V2. Please set the configuration spark.sql.storeAssignmentPolicy to other values.
[info] at org.apache.spark.sql.errors.QueryCompilationErrors$.legacyStoreAssignmentPolicyError(QueryCompilationErrors.scala:270)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$.org$apache$spark$sql$catalyst$analysis$ResolveRowLevelCommandAssignments$$validateStoreAssignmentPolicy(ResolveRowLevelCommandAssignments.scala:66)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$$anonfun$apply$2.applyOrElse(ResolveRowLevelCommandAssignments.scala:52)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$$anonfun$apply$2.applyOrElse(ResolveRowLevelCommandAssignments.scala:41)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsDownWithPruning$2(AnalysisHelper.scala:170)
[info] at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:76)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsDownWithPruning$1(AnalysisHelper.scala:170)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.allowInvokingTransformsInAnalyzer(AnalysisHelper.scala:323)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsDownWithPruning(AnalysisHelper.scala:168)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsDownWithPruning$(AnalysisHelper.scala:164)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsDownWithPruning(LogicalPlan.scala:32)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsWithPruning(AnalysisHelper.scala:99)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsWithPruning$(AnalysisHelper.scala:96)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsWithPruning(LogicalPlan.scala:32)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$.apply(ResolveRowLevelCommandAssignments.scala:41)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$.apply(ResolveRowLevelCommandAssignments.scala:38)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org