You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Vinay (Jira)" <ji...@apache.org> on 2021/06/08 12:48:00 UTC
[jira] [Assigned] (HUDI-1892) NullPointerException when using
OverwriteNonDefaultsWithLatestAvroPayload at hudi 0.9.0
[ https://issues.apache.org/jira/browse/HUDI-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay reassigned HUDI-1892:
---------------------------
Assignee: Vinay
> NullPointerException when using OverwriteNonDefaultsWithLatestAvroPayload at hudi 0.9.0
> ---------------------------------------------------------------------------------------
>
> Key: HUDI-1892
> URL: https://issues.apache.org/jira/browse/HUDI-1892
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: shenbing
> Assignee: Vinay
> Priority: Major
>
> using compiled hudi 0.9.0 with hadoop3.0.0 and hive3.1.1 after resolving dependency conflicts,
> I import hudi-spark-bundle_2.11-0.9.0-SNAPSHOT.jar into my project.
> When I using OverwriteNonDefaultsWithLatestAvroPayload to update field with new value, I got the error.
> {code:java}
> Caused by: java.util.concurrent.ExecutionException: org.apache.hudi.exception.HoodieUpsertException: Failed to combine/merge new record with old value in storage, for new record {HoodieRecord{key=HoodieKey { recordKey=1 partitionPath=date=1}, currentLocation='HoodieRecordLocation {instantTime=20210510160355, fileId=9a0fcb8e-8cd9-4c9c-bea8-46bbf509035e-0}', newLocation='HoodieRecordLocation {instantTime=20210510160400, fileId=9a0fcb8e-8cd9-4c9c-bea8-46bbf509035e-0}'}}, old value {{"_hoodie_commit_time": "20210510160355", "_hoodie_commit_seqno": "20210510160355_0_50", "_hoodie_record_key": "1", "_hoodie_partition_path": "date=1", "_hoodie_file_name": "9a0fcb8e-8cd9-4c9c-bea8-46bbf509035e-0_0-1502-1519_20210510160355.parquet", "uuid": "1", "name": "jerry", "age": 10, "date": "1", "update_time": "1"}}
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.hudi.common.util.queue.BoundedInMemoryExecutor.execute(BoundedInMemoryExecutor.java:141)
> ... 34 more
> Caused by: org.apache.hudi.exception.HoodieUpsertException: Failed to combine/merge new record with old value in storage, for new record {HoodieRecord{key=HoodieKey { recordKey=1 partitionPath=date=1}, currentLocation='HoodieRecordLocation {instantTime=20210510160355, fileId=9a0fcb8e-8cd9-4c9c-bea8-46bbf509035e-0}', newLocation='HoodieRecordLocation {instantTime=20210510160400, fileId=9a0fcb8e-8cd9-4c9c-bea8-46bbf509035e-0}'}}, old value {{"_hoodie_commit_time": "20210510160355", "_hoodie_commit_seqno": "20210510160355_0_50", "_hoodie_record_key": "1", "_hoodie_partition_path": "date=1", "_hoodie_file_name": "9a0fcb8e-8cd9-4c9c-bea8-46bbf509035e-0_0-1502-1519_20210510160355.parquet", "uuid": "1", "name": "jerry", "age": 10, "date": "1", "update_time": "1"}}
> at org.apache.hudi.io.HoodieMergeHandle.write(HoodieMergeHandle.java:290)
> at org.apache.hudi.table.action.commit.AbstractMergeHelper$UpdateHandler.consumeOneRecord(AbstractMergeHelper.java:122)
> at org.apache.hudi.table.action.commit.AbstractMergeHelper$UpdateHandler.consumeOneRecord(AbstractMergeHelper.java:112)
> at org.apache.hudi.common.util.queue.BoundedInMemoryQueueConsumer.consume(BoundedInMemoryQueueConsumer.java:37)
> at org.apache.hudi.common.util.queue.BoundedInMemoryExecutor.lambda$null$2(BoundedInMemoryExecutor.java:121)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> Caused by: java.lang.NullPointerException
> at org.apache.hudi.common.model.OverwriteWithLatestAvroPayload.overwriteField(OverwriteWithLatestAvroPayload.java:97)
> at org.apache.hudi.common.model.OverwriteNonDefaultsWithLatestAvroPayload.lambda$combineAndGetUpdateValue$0(OverwriteNonDefaultsWithLatestAvroPayload.java:67)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at org.apache.hudi.common.model.OverwriteNonDefaultsWithLatestAvroPayload.combineAndGetUpdateValue(OverwriteNonDefaultsWithLatestAvroPayload.java:64)
> at org.apache.hudi.common.model.HoodieRecordPayload.combineAndGetUpdateValue(HoodieRecordPayload.java:81)
> at org.apache.hudi.io.HoodieMergeHandle.write(HoodieMergeHandle.java:276)
> ... 8 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)