[GitHub] [iceberg] hililiwei commented on pull request #7217: Flink: Resolve writers object of PartitionedDeltaWriter will cause OOM when partition number is big

["hililiwei (via GitHub)" <gi...@apache.org>]

hililiwei commented on PR #7217:
URL: https://github.com/apache/iceberg/pull/7217#issuecomment-1495426143

   
   > @hililiwei I modify PartitionedDeltaWriter#close method to paralle close. It can reduce checkpoint time.
   > 
   > ```java
   >  Tasks.foreach(writers.values())
   >                     .throwFailureWhenFinished()
   >                     .noRetry()
   >                     .executeWith(EXECUTOR_SERVICE) // set a ExecutorService object
   >                     .run(RowDataDeltaWriter::close, IOException.class);
   > ```
   
   I'm glad you worked it out. Writers of different partitions can be closed in parallel, which I think is OK.
   Good luck.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org