You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@uniffle.apache.org by GitBox <gi...@apache.org> on 2022/07/27 10:25:58 UTC

[GitHub] [incubator-uniffle] zuston opened a new issue, #90: [Performance Optimization] Improve the speed of writing index file in shuffle server

zuston opened a new issue, #90:
URL: https://github.com/apache/incubator-uniffle/issues/90

   ### Motivation
   When I test uniffle performance, i found a huge performance drop due to the low speed of writing index file. Flame graph attached:
   
   ![reliao_img_1658917352873](https://user-images.githubusercontent.com/8609142/181224611-faf20e2b-a65a-44ad-ad15-560addf43c85.png)
   
   ### Solution
   Use the `dataOutputStream = new DataOutputStream(new BufferedOutputStream(fileOutputStream));` in `LocalFileWriter`. Please refer to: https://stackoverflow.com/questions/38439410/java-dataoutputstream-writelong-byte-double-int-speed
   
   ### Result
   
   After apply this optimization:
   ![reliao_img_1658917526994](https://user-images.githubusercontent.com/8609142/181225186-b883658b-ac40-40a2-8cb7-22e154bc3d7f.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] colinmjj commented on issue #90: [Performance Optimization] Improve the speed of writing index file in shuffle server

Posted by GitBox <gi...@apache.org>.

colinmjj commented on issue #90:
URL: https://github.com/apache/incubator-uniffle/issues/90#issuecomment-1196587758

   @zuston good catch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi commented on issue #90: [Performance Optimization] Improve the speed of writing index file in shuffle server

Posted by GitBox <gi...@apache.org>.

jerqi commented on issue #90:
URL: https://github.com/apache/incubator-uniffle/issues/90#issuecomment-1197592448

   solved by #91 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi closed issue #90: [Performance Optimization] Improve the speed of writing index file in shuffle server

Posted by GitBox <gi...@apache.org>.

jerqi closed issue #90: [Performance Optimization] Improve the speed of writing index file in shuffle server
URL: https://github.com/apache/incubator-uniffle/issues/90


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org