[GitHub] [hudi] yanghua commented on a change in pull request #2143: [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable

[GitBox <gi...@apache.org>]

yanghua commented on a change in pull request #2143:
URL: https://github.com/apache/hudi/pull/2143#discussion_r502103169



##########
File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/HoodieWriteableTestTable.java
##########
@@ -94,6 +106,10 @@ public String withInserts(String partition) throws Exception {
   }
 
   public String withInserts(String partition, HoodieRecord... records) throws Exception {
+    return withInserts(partition, Arrays.asList(records));
+  }
+
+  public String withInserts(String partition, List<HoodieRecord> records) throws Exception {

Review comment:
       Considering the fluent APIs have one general pattern: method `withXXX` returns `HoodieWriteableTestTable ` itself. However, this method and above break this rule. WDYT about renaming them to `getFileIdWithInserts` or `returnFileIdWithInserts`?

##########
File path: hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/testutils/HoodieWriteableTestTable.java
##########
@@ -128,4 +148,37 @@ public HoodieWriteableTestTable withInserts(String partition, String fileId, Hoo
 
     return this;
   }
+
+  public HoodieWriteableTestTable withLogAppends(HoodieRecord... records) throws Exception {
+    return withLogAppends(Arrays.asList(records));
+  }
+
+  public HoodieWriteableTestTable withLogAppends(List<HoodieRecord> records) throws Exception {
+    for (List<HoodieRecord> groupedRecords: records.stream()
+        .collect(Collectors.groupingBy(HoodieRecord::getCurrentLocation)).values()) {
+      appendRecordsToLogFile(groupedRecords);
+    }
+    return this;
+  }
+
+  private void appendRecordsToLogFile(List<HoodieRecord> groupedRecords) throws Exception {
+    String partitionPath = groupedRecords.get(0).getPartitionPath();
+    HoodieRecordLocation location = groupedRecords.get(0).getCurrentLocation();
+    try (HoodieLogFormat.Writer logWriter = HoodieLogFormat.newWriterBuilder().onParentPath(new Path(basePath, partitionPath))
+        .withFileExtension(HoodieLogFile.DELTA_EXTENSION).withFileId(location.getFileId())
+        .overBaseCommit(location.getInstantTime()).withFs(fs).build()) {
+      Map<HoodieLogBlock.HeaderMetadataType, String> header = new HashMap<>();
+      header.put(HoodieLogBlock.HeaderMetadataType.INSTANT_TIME, location.getInstantTime());
+      header.put(HoodieLogBlock.HeaderMetadataType.SCHEMA, schema.toString());
+      logWriter.appendBlock(new HoodieAvroDataBlock(groupedRecords.stream().map(r -> {
+        try {
+          GenericRecord val = (GenericRecord) r.getData().getInsertValue(schema).get();
+          HoodieAvroUtils.addHoodieKeyToRecord(val, r.getRecordKey(), r.getPartitionPath(), "");
+          return (IndexedRecord) val;
+        } catch (IOException e) {

Review comment:
       Logging the detailed exception information looks better?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org