You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by si...@apache.org on 2022/01/07 18:06:36 UTC

[hudi] 14/21: [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when records by keys (#4505)

This is an automated email from the ASF dual-hosted git repository.

sivabalan pushed a commit to branch release-0.10.1-rc1
in repository https://gitbox.apache.org/repos/asf/hudi.git

commit f91b3f348f3d283f28cc5b12751ce1d3952e684a
Author: Manoj Govindassamy <ma...@gmail.com>
AuthorDate: Tue Jan 4 13:41:33 2022 -0800

    [HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when records by keys (#4505)
    
    - HoodieMetadataMergedLogRecordReader#getRecordsByKeys() and its parent class methods
       are not thread safe. When multiple queries come in for gettting log records
       by keys, they all operate on the same log record reader instance provided by
       HoodieBackedTableMetadata#openReadersIfNeeded() and they trip over each other
       as they clear/put/get the same class memeber records.
    
     - The fix is to streamline the mutatation to class member records. Making
       HoodieMetadataMergedLogRecordReader#getRecordsByKeys() a synchronized method
    to avoid concurrent log records readers getting into NPE.
---
 .../apache/hudi/metadata/HoodieMetadataMergedLogRecordReader.java    | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataMergedLogRecordReader.java b/hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataMergedLogRecordReader.java
index e635eea..01c8d05 100644
--- a/hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataMergedLogRecordReader.java
+++ b/hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataMergedLogRecordReader.java
@@ -120,7 +120,10 @@ public class HoodieMetadataMergedLogRecordReader extends HoodieMergedLogRecordSc
     return Collections.singletonList(Pair.of(key, Option.ofNullable((HoodieRecord) records.get(key))));
   }
 
-  public List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>> getRecordsByKeys(List<String> keys) {
+  public synchronized List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>> getRecordsByKeys(List<String> keys) {
+    // Following operations have to be atomic, otherwise concurrent
+    // readers would race with each other and could crash when
+    // processing log block records as part of scan.
     records.clear();
     scan(Option.of(keys));
     List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>> metadataRecords = new ArrayList<>();