[GitHub] [kafka] vvcephei commented on pull request #9070: MINOR: speed up release script

[GitBox <gi...@apache.org>]

vvcephei commented on pull request #9070:
URL: https://github.com/apache/kafka/pull/9070#issuecomment-663321224


   Hey all, after waiting at least 10 hours cumulatively for file uploads (so far) for the 2.5.1 release candidates, I've been feeling motivated to try and improve the situation.
   
   Currently, it takes about 2 hours to upload the release artifacts, and almost all of that time is just establishing an ssh connection over and over again. Each connection takes about 6 seconds to establish, and then a few milliseconds to transfer the file.
   
   I was a little surprised to see this since it looked like the code was supposed to be using sftp in batch mode, but was actually doing a batch of one command each time.
   
   I found in https://github.com/apache/kafka/pull/5374 that this was actually intentional, because the full batch of commands was taking too long and making people think the command had hung.
   
   I'm proposing to instead expand the toolchain to use `tar`, `rsync`, and `ssh`. These seem like reasonable requirements, and I've double-checked the commands on a Mac, and they seem to do the same thing as they do on Linux.
   
   I don't know how long the batch upload was taking before, but the rsyncing the compressed artifact is taking me about one minute. Considering the compilation step a little earlier "hangs" with no output for at least two minutes, this seems to be ok.
   
   With this change, the release script takes about five minutes to run instead of two hours.
   
   WDYT?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org