[GitHub] [hudi] nsivabalan commented on a change in pull request #3149: [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows

[GitBox <gi...@apache.org>]

nsivabalan commented on a change in pull request #3149:
URL: https://github.com/apache/hudi/pull/3149#discussion_r664603586



##########
File path: hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/internal/BulkInsertDataInternalWriterHelper.java
##########
@@ -92,19 +95,30 @@ public void write(InternalRow record) throws IOException {
   public void abort() {
   }
 
-  private void createNewHandle(String partitionPath) throws IOException {
-    if (null != handle) {
-      close();
+  private HoodieRowCreateHandle getRowCreateHandle(String partitionPath) throws IOException {
+    if (!handles.containsKey(partitionPath)) { // if there is no handle corresponding to the partition path
+      // if records are sorted, we can close all existing handles
+      if (writeConfig.getBulkInsertIsPartitionRecordsSorted()) {

Review comment:
       here is the usage of arcPartitionRecordsSorted. basically, whether to cache and reuse the write handles or close it right away when switching to a diff partition. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org