You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2019/07/01 18:46:51 UTC
[GitHub] [incubator-hudi] ovj opened a new issue #773: Updates to
HoodieRecord are missing during MergeHandle with ExternalSpillableMap
ovj opened a new issue #773: Updates to HoodieRecord are missing during MergeHandle with ExternalSpillableMap
URL: https://github.com/apache/incubator-hudi/issues/773
MergeHandle uses ExternalSpillableMap for storing all new inserts to a file. Today during merge handle if number of records which we are adding to a file are a lot then ExternalSpillableMap may end up writing it to disk and future updates done to in memory copy of HoodieRecord will not be reflected into on disk copy. This creates duplicate key issue if we are using HBaseIndex. Creating this issue to address this.
https://github.com/apache/incubator-hudi/blob/master/hoodie-client/src/main/java/com/uber/hoodie/io/HoodieMergeHandle.java#L211 Here we are updating HoodieRecord after adding to the map.
cc @vinothchandar @bvaradar @n3nash
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services