[PR] Committer memory optimization [paimon]

["codeTai (via GitHub)" <gi...@apache.org>]

codeTai opened a new pull request, #3592:
URL: https://github.com/apache/paimon/pull/3592

   
   ### Purpose
   
   <!-- Linking this pull request to the issue -->
   Linked issue: close https://github.com/apache/paimon/issues/3590
   
   <!-- What is the purpose of the change -->
   When submitting a snapshot triggers a full compaction of the manifest file, we hope to reduce the usage of the taskManager heap memory.
   
   ### Tests
   
   <!-- List UT and IT cases to verify this change -->
   Before the comparison test, fine-grained-resource-management has been enabled，The committer operator runs in a separate task manager.
   
   Heap memory usage before optimization：
   
   ![image](https://github.com/apache/paimon/assets/20182632/21184999-26c1-43cc-994d-2d98734e0c0c)
   
   Heap memory usage after optimization：
   ![image](https://github.com/apache/paimon/assets/20182632/e2106b79-0fc1-412d-8cf3-bc5713a8e780)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org