[jira] [Commented] (HUDI-389) Updates sent to diff partition for a given key with Global Index

["sivabalan narayanan (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991120#comment-16991120 ] 

sivabalan narayanan commented on HUDI-389:
------------------------------------------

[~vinothchandar] When I tried creating a new HoodieRecord, I got compilation error if I don't set type. Will try once to cross check once again. thanks. 

> Updates sent to diff partition for a given key with Global Index 
> -----------------------------------------------------------------
>
>                 Key: HUDI-389
>                 URL: https://issues.apache.org/jira/browse/HUDI-389
>             Project: Apache Hudi (incubating)
>          Issue Type: Bug
>          Components: Index
>            Reporter: sivabalan narayanan
>            Assignee: sivabalan narayanan
>            Priority: Major
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> Updates sent to diff partition for a given key with Global Index should succeed by updating the record under original partition. As of now, it throws exception. 
> error log:
> 14738 [Executor task launch worker-0] INFO com.uber.hoodie.common.table.timeline.HoodieActiveTimeline - Loaded instants java.util.stream.ReferencePipeline$Head@d02b1c7
> 14738 [Executor task launch worker-0] INFO com.uber.hoodie.common.table.view.AbstractTableFileSystemView - Building file system view for partition (2016/04/15)
> 14738 [Executor task launch worker-0] INFO com.uber.hoodie.common.table.view.AbstractTableFileSystemView - #files found in partition (2016/04/15) =0, Time taken =0
> 14738 [Executor task launch worker-0] INFO com.uber.hoodie.common.table.view.AbstractTableFileSystemView - addFilesToView: NumFiles=0, FileGroupsCreationTime=0, StoreTimeTaken=0
> 14738 [Executor task launch worker-0] INFO com.uber.hoodie.common.table.view.HoodieTableFileSystemView - Adding file-groups for partition :2016/04/15, #FileGroups=0
> 14738 [Executor task launch worker-0] INFO com.uber.hoodie.common.table.view.AbstractTableFileSystemView - Time to load partition (2016/04/15) =0
> 14754 [Executor task launch worker-0] ERROR com.uber.hoodie.table.HoodieCopyOnWriteTable - Error upserting bucketType UPDATE for partition :0
> java.util.NoSuchElementException: No value present
> at com.uber.hoodie.common.util.Option.get(Option.java:112)
> at com.uber.hoodie.io.HoodieMergeHandle.(HoodieMergeHandle.java:71)
> at com.uber.hoodie.table.HoodieCopyOnWriteTable.getUpdateHandle(HoodieCopyOnWriteTable.java:226)
> at com.uber.hoodie.table.HoodieCopyOnWriteTable.handleUpdate(HoodieCopyOnWriteTable.java:180)
> at com.uber.hoodie.table.HoodieCopyOnWriteTable.handleUpsertPartition(HoodieCopyOnWriteTable.java:263)
> at com.uber.hoodie.HoodieWriteClient.lambda$upsertRecordsInternal$7ef77fd$1(HoodieWriteClient.java:442)
> at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102)
> at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102)
> at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)
> at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:336)
> at org.apache.spark.rdd.RDD$$anonfun$8.apply(RDD.scala:334)
> at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:973)
> at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:948)
> at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:888)
> at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
> at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:694)
> at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:334)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:285)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
> at org.apache.spark.scheduler.Task.run(Task.scala:99)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>  
> Refer [here|[https://github.com/apache/incubator-hudi/issues/1021]] for some context. 
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)