You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2023/12/02 12:59:01 UTC
[PR] [SPARK-45975][TESTS][3.5] Reset storeAssignmentPolicy to original [spark]
LuciferYang opened a new pull request, #44126:
URL: https://github.com/apache/spark/pull/44126
### What changes were proposed in this pull request?
Reset storeAssignmentPolicy to original in HiveCompatibilitySuite.
### Why are the changes needed?
STORE_ASSIGNMENT_POLICY was not reset in HiveCompatibilitySuite, causing subsequent test cases to fail.
Details: https://github.com/wForget/spark/actions/runs/6902668865/job/18779862759
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GitHub Actions
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45975][SQL][TESTS][3.5] Reset storeAssignmentPolicy to original [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #44126:
URL: https://github.com/apache/spark/pull/44126#issuecomment-1837230461
Ack, @LuciferYang .
- Did you see the same failure in `master` branch?
- If this doesn't help `branch-3.5`, we can simply close this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45975][SQL][TESTS][3.5] Reset storeAssignmentPolicy to original [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #44126:
URL: https://github.com/apache/spark/pull/44126#issuecomment-1837234889
Merged to branch-3.5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45975][SQL][TESTS][3.5] Reset storeAssignmentPolicy to original [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #44126: [SPARK-45975][SQL][TESTS][3.5] Reset storeAssignmentPolicy to original
URL: https://github.com/apache/spark/pull/44126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45975][SQL][TESTS][3.5] Reset storeAssignmentPolicy to original [spark]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget commented on PR #44126:
URL: https://github.com/apache/spark/pull/44126#issuecomment-1837692534
Sorry for not replying in time during the holiday, and thanks to @LuciferYang for this fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45975][TESTS][3.5] Reset storeAssignmentPolicy to original [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #44126:
URL: https://github.com/apache/spark/pull/44126#issuecomment-1837143832
- https://github.com/apache/spark/actions/runs/7041075997/job/19163039661
- https://github.com/apache/spark/actions/runs/7054970010/job/19204693143
```
[info] HiveSourceRowLevelOperationSuite:
03:30:31.518 WARN org.apache.spark.sql.catalyst.analysis.ResolveSessionCatalog: A Hive serde table will be created as there is no table provider specified. You can set spark.sql.legacy.createHiveTableByDefault to false so that native data source table will be created instead.
03:30:31.535 WARN org.apache.hadoop.hive.metastore.HiveMetaStore: Location: file:/home/runner/work/spark/spark/target/tmp/warehouse-b721773d-f779-4db4-9ebc-ca8315b15f61/hive_table specified for non-external table:hive_table
[info] - SPARK-45943: merge into using hive table without stats *** FAILED *** (391 milliseconds)
[info] org.apache.spark.sql.AnalysisException: LEGACY store assignment policy is disallowed in Spark data source V2. Please set the configuration spark.sql.storeAssignmentPolicy to other values.
[info] at org.apache.spark.sql.errors.QueryCompilationErrors$.legacyStoreAssignmentPolicyError(QueryCompilationErrors.scala:270)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$.org$apache$spark$sql$catalyst$analysis$ResolveRowLevelCommandAssignments$$validateStoreAssignmentPolicy(ResolveRowLevelCommandAssignments.scala:66)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$$anonfun$apply$2.applyOrElse(ResolveRowLevelCommandAssignments.scala:52)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$$anonfun$apply$2.applyOrElse(ResolveRowLevelCommandAssignments.scala:41)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsDownWithPruning$2(AnalysisHelper.scala:170)
[info] at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:76)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.$anonfun$resolveOperatorsDownWithPruning$1(AnalysisHelper.scala:170)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.allowInvokingTransformsInAnalyzer(AnalysisHelper.scala:323)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsDownWithPruning(AnalysisHelper.scala:168)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsDownWithPruning$(AnalysisHelper.scala:164)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsDownWithPruning(LogicalPlan.scala:32)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsWithPruning(AnalysisHelper.scala:99)
[info] at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.resolveOperatorsWithPruning$(AnalysisHelper.scala:96)
[info] at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsWithPruning(LogicalPlan.scala:32)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$.apply(ResolveRowLevelCommandAssignments.scala:41)
[info] at org.apache.spark.sql.catalyst.analysis.ResolveRowLevelCommandAssignments$.apply(ResolveRowLevelCommandAssignments.scala:38)
```
backport to fix branch-3.5 test fail, cc @wForget @cloud-fan @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45975][SQL][TESTS][3.5] Reset storeAssignmentPolicy to original [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #44126:
URL: https://github.com/apache/spark/pull/44126#issuecomment-1837232251
@dongjoon-hyun
For the master branch, this pr was merged before https://github.com/apache/spark/pull/43867, so the test fail will not be reproduced. But for branch-3.5, due to the no this patch, the CI has encountered a test failure as https://github.com/apache/spark/pull/44126#issuecomment-1837143832
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45975][SQL][TESTS][3.5] Reset storeAssignmentPolicy to original [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #44126:
URL: https://github.com/apache/spark/pull/44126#issuecomment-1837235615
Thanks @dongjoon-hyun ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org