[5/7] git commit: Port "Using Libcloud in multi-threaded and async environments" documentation section.

[to...@apache.org]

Port "Using Libcloud in multi-threaded and async environments" documentation
section.


Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo
Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/b6be3e70
Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/b6be3e70
Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/b6be3e70

Branch: refs/heads/trunk
Commit: b6be3e70e1fe31281f537c74c9b435e52264e20c
Parents: 318ed78
Author: Tomaz Muraus <to...@apache.org>
Authored: Sat Aug 3 17:52:50 2013 +0200
Committer: Tomaz Muraus <to...@apache.org>
Committed: Sat Aug 3 17:52:50 2013 +0200

----------------------------------------------------------------------
 docs/examples/misc/twisted_create_node.py       | 34 ++++++++++
 docs/index.rst                                  |  3 +-
 ...-in-multithreaded-and-async-environments.rst | 69 ++++++++++++++++++++
 3 files changed, 105 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/libcloud/blob/b6be3e70/docs/examples/misc/twisted_create_node.py
----------------------------------------------------------------------
diff --git a/docs/examples/misc/twisted_create_node.py b/docs/examples/misc/twisted_create_node.py
new file mode 100644
index 0000000..e0109e1
--- /dev/null
+++ b/docs/examples/misc/twisted_create_node.py
@@ -0,0 +1,34 @@
+from __future__ import absolute_import
+
+from pprint import pprint
+
+from twisted.internet import defer, threads, reactor
+from libcloud.compute.types import Provider
+from libcloud.compute.providers import get_driver
+
+
+@defer.inlineCallbacks
+def create_node(name):
+    node = yield threads.deferToThread(_thread_create_node,
+                                       name=name)
+    pprint(node)
+    reactor.stop()
+
+
+def _thread_create_node(name):
+    Driver = get_driver(Provider.RACKSPACE)
+    conn = Driver('username', 'api key')
+    image = conn.list_images()[0]
+    size = conn.list_sizes()[0]
+    node = conn.create_node(name=name, image=image, size=size)
+    return node
+
+
+def stop(*args, **kwargs):
+    reactor.stop()
+
+d = create_node(name='my-lc-node')
+d.addCallback(stop)
+d.addErrback(stop)
+
+reactor.run()

http://git-wip-us.apache.org/repos/asf/libcloud/blob/b6be3e70/docs/index.rst
----------------------------------------------------------------------
diff --git a/docs/index.rst b/docs/index.rst
index 09cff04..be5f382 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -23,14 +23,15 @@ resources:
 * :doc:`Load Balancers as a Service </loadbalancer/index>`
 * :doc:`DNS as a Service </dns/index>`
 
-
 .. toctree::
     :glob:
+    :maxdepth: 2
     :hidden:
 
     compute/*
     storage/*
     loadbalancer/*
     dns/*
+    other/*
 
 .. _`Apache 2.0 license`: https://www.apache.org/licenses/LICENSE-2.0.html

http://git-wip-us.apache.org/repos/asf/libcloud/blob/b6be3e70/docs/other/using-libcloud-in-multithreaded-and-async-environments.rst
----------------------------------------------------------------------
diff --git a/docs/other/using-libcloud-in-multithreaded-and-async-environments.rst b/docs/other/using-libcloud-in-multithreaded-and-async-environments.rst
new file mode 100644
index 0000000..26e1edb
--- /dev/null
+++ b/docs/other/using-libcloud-in-multithreaded-and-async-environments.rst
@@ -0,0 +1,69 @@
+Using Libcloud in multi-threaded and async environments
+=======================================================
+
+Libcloud's primary task is to communicate with different provider APIs using
+HTTP. This means most of the work is not CPU intensive, but performing all
+those HTTP requests includes a lot of waiting which makes the library I/O
+bound.
+
+Most of the time you want to perform more operations in parallel or just
+want your code to finish faster (for example starting a lot of servers or
+periodically polling for node status).
+
+Problems like this are usually solved using threads or async libraries such
+as Twisted, Tornado or gevent.
+
+This page contains some information and tips about how to use Libcloud in
+such environments.
+
+Libcloud and thread-safety
+--------------------------
+
+Important thing to keep in mind when dealing with threads is thread-safety.
+Libcloud driver instance is **not** thread safe. This means if you don't want
+to deal with complex (and usually inefficient) locking the easiest solution
+is to create a new driver instance inside each thread.
+
+Using Libcloud with gevent
+--------------------------
+
+gevent has an ability to monkey patch and replace functions in the Python
+``socket``, ``urllib2``, ``httplib`` and ``time`` module with its own
+functions which don't block.
+
+You need to do two things when you want to use Libcloud with gevent:
+
+* Enable monkey patching
+
+.. sourcecode:: python
+
+    from gevent import monkey
+    monkey.patch_all()
+
+* Create a separate driver instance for each Greenlet. This is necessary
+  because a driver instance reuses the same Connection class.
+
+For an example see Efficiently download multiple files using gevent.
+
+Using Libcloud with Twisted
+---------------------------
+
+Libcloud has no Twisted support included in the core which means you need
+to be careful when you use it with Twisted and some other async frameworks.
+
+If you don't use it properly it can block the whole reactor (similar as
+any other blocking library or a long CPU-intensive task) which means the
+execution of other pending tasks in the event queue will be blocked.
+
+A simple solution to prevent blocking the reactor is to run Libcloud
+calls inside a thread. In Twisted this can be achieved using
+``threads.deferToThread`` which runs a provided method inside the Twisted
+thread pool.
+
+The example bellow demonstrates how to create a new node inside a thread
+without blocking the whole reactor.
+
+.. literalinclude:: /examples/misc/twist_create_node.py
+   :language: python
+
+