[GitHub] [incubator-celeborn] waitinfuture commented on a diff in pull request #1519: [CELEBORN-617] [FLINK] MapPartitionFileWriter updates flushing filele…

["waitinfuture (via GitHub)" <gi...@apache.org>]

waitinfuture commented on code in PR #1519:
URL: https://github.com/apache/incubator-celeborn/pull/1519#discussion_r1209678001


##########
common/src/main/java/org/apache/celeborn/common/meta/FileInfo.java:
##########
@@ -110,6 +110,11 @@ public synchronized long getFileLength() {
     return chunkOffsets.get(chunkOffsets.size() - 1);
   }
 
+  // for mappartition, a file only has one chunk.
+  public synchronized void updateFileLength(long fileLength) {

Review Comment:
   I think we'd better to set chunk size to infinity for map partition to avoid writing chunk offsets, instead of introducing this branch logic.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org