[GitHub] [libcloud] Kami opened a new pull request #1483: ParamikoSSHClient file upload performance improvements and other ParamikoSSHClient improvements

[GitBox <gi...@apache.org>]

Kami opened a new pull request #1483:
URL: https://github.com/apache/libcloud/pull/1483


   This pull request includes the following improvements to the ``ParamikoSSHClient`` client and related ``deploy_node()`` functionality which utilizes paramiko SSH client.
   
   1. ``put()`` method now tries to re-use the existing sftp connection (if one is already opened) for successive calls to ``ssh_client.put()`` method.
   
   This should speed up ``deploy_node()`` functionality when uploading multiple files (aka using multiple ``FileDeployment`` steps), because it means we don't need to close and re-open sftp connection for each call to ``put()`` method.
   
   2. Add new ``keep_alive`` and ``use_compression`` arguments to the ``ParamikoSSHClient`` class constructor. At this point, those arguments are not exposed directly to the ``deploy_node()`` method yet.
   
   3. Add new ``putfo()`` method to the ``ParamikoSSHClient`` class and update ``FileDeployment.run()`` method to use this method instead of ``put()``.
   
   This method is much more efficient than ``put()`` because it means we don't need to buffer the whole file content in memory when calling ``file_deployment_step.run()`` and we can also utilize pipelining.
   
   Keep in mind that ``FileDeployment`` was primarily meant for uploading small script files and not large binary files, but based on testing, this change substantially increases performance when uploading larger binary files using ``FileDeployment`` class (I've seen speed ups of up to 80% in favor of the new implementation).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] Kami merged pull request #1483: ParamikoSSHClient file upload performance improvements and other ParamikoSSHClient improvements

[GitBox <gi...@apache.org>]

Kami merged pull request #1483:
URL: https://github.com/apache/libcloud/pull/1483


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org