[GitHub] [storm] StaticMian opened a new pull request #3013: fix uploading dependency jars too slow when storm client and server located in different IDC

[GitBox <gi...@apache.org>]

StaticMian opened a new pull request #3013: fix uploading dependency jars too slow when storm client and server located in different IDC
URL: https://github.com/apache/storm/pull/3013
 
 
   when storm client and server is locating in different IDC(one is in Beijing, while another in Shanghai), uploading dependency jars may take a very long long time(in my case, 31minutes!)...
   
       when I digged into this, I found that in DependencyUploader,  method "uploadDependencyToBlobStore" using JDK NIO's Files.copy to upload local jars to remote Blob server. In Files.copy(Path, OutputStream), the buffer size is 8k by default, given that latency between Beijing and Shanghai is about 20ms, a dependency fat jar of 360M finally cost me 'a lunch time' to finish uploading!!!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services