GCP HeavyWeightReadGroup API requests

[Poul Petersen <pe...@bigml.com>]

  Hello:

  I recently added support for Libcloud to our internal infrastructure
management tool as part of a project to support GCP (and other providers).
We are currently managing a deployment of 50 or so instances which we plan
to grow to several hundred.

  In order to optimize costs, we have several custom threads which monitor
usage and power up/down instances when they are needed. This is managed
with several Python threads which are running in parallel. Most of the
application state is kept locally to reduce API calls, but there are a few
things that we rely on the cloud API to provide us, in particular the
instance state (running/stopped/etc).

  The issue we ran into is that we restarted these threads and they all
went into a cycle of calling ex_get_node() for each instance to refresh the
state and other instance metadata. This quickly swamped our API quota for
GCP, specifically the HeavyWeightReadGroup request.

  We have since made some changes to avoid this scenario and to reduce the
number of calls we make (and we got our quota increased), but as we eye
bigger deployments I'm wondering if there is a better way? If I only need
to refresh the instance.state, is there a different API call I can make? I
couldn't find anything other than to rebuild my local object with a call to
ex_get_node(). For example, boto's EC2 Instance class has an update()
method which refreshes the state and tags - Is there something similar in
Libcloud?

Thanks,

-poul