You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Alexey Kudinkin (Jira)" <ji...@apache.org> on 2022/04/22 02:05:00 UTC
[jira] [Closed] (HUDI-3936) When precombine field is nested, MOR query breaks
[ https://issues.apache.org/jira/browse/HUDI-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexey Kudinkin closed HUDI-3936.
---------------------------------
Resolution: Fixed
> When precombine field is nested, MOR query breaks
> -------------------------------------------------
>
> Key: HUDI-3936
> URL: https://issues.apache.org/jira/browse/HUDI-3936
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark
> Reporter: sivabalan narayanan
> Assignee: Ethan Guo
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.11.0
>
>
> For MOR table, when using a nested field as precombined key, when querying the table after upserts, the following exception is thrown:
>
> {code:java}
> Exception in thread "main" java.util.NoSuchElementException: key not found: meta.version
> at scala.collection.MapLike.default(MapLike.scala:236)
> at scala.collection.MapLike.default$(MapLike.scala:235)
> at scala.collection.AbstractMap.default(Map.scala:65)
> at scala.collection.MapLike.apply(MapLike.scala:144)
> at scala.collection.MapLike.apply$(MapLike.scala:143)
> at scala.collection.AbstractMap.apply(Map.scala:65)
> at org.apache.hudi.HoodieSparkUtils$.$anonfun$getRequiredSchema$2(HoodieSparkUtils.scala:325)
> at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
> at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
> at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
> at scala.collection.TraversableLike.map(TraversableLike.scala:286)
> at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
> at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:198)
> at org.apache.hudi.HoodieSparkUtils$.getRequiredSchema(HoodieSparkUtils.scala:325)
> at org.apache.hudi.HoodieBaseRelation.buildScan(HoodieBaseRelation.scala:219)
> at org.apache.spark.sql.execution.datasources.DataSourceStrategy$.$anonfun$apply$4(DataSourceStrategy.scala:332)
> at org.apache.spark.sql.execution.datasources.DataSourceStrategy$.$anonfun$pruneFilterProject$1(DataSourceStrategy.scala:365)
> at org.apache.spark.sql.execution.datasources.DataSourceStrategy$.pruneFilterProjectRaw(DataSourceStrategy.scala:442)
> at org.apache.spark.sql.execution.datasources.DataSourceStrategy$.pruneFilterProject(DataSourceStrategy.scala:364)
> at org.apache.spark.sql.execution.datasources.DataSourceStrategy$.apply(DataSourceStrategy.scala:332)
> at org.apache.spark.sql.catalyst.planning.QueryPlanner.$anonfun$plan$1(QueryPlanner.scala:63)
> at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:486)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:492)
> at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491) {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)