[jira] [Created] (JCLOUDS-490) Security group listing on nova slow

["Simon Hildrew (JIRA)" <ji...@apache.org>]

Simon Hildrew created JCLOUDS-490:
-------------------------------------

             Summary: Security group listing on nova slow
                 Key: JCLOUDS-490
                 URL: https://issues.apache.org/jira/browse/JCLOUDS-490
             Project: jclouds
          Issue Type: Bug
    Affects Versions: 1.7.1, 1.7.0
         Environment: Linux, Scala, Openstack
            Reporter: Simon Hildrew


Calls to retrieve the security groups in an Openstack tenant using the jClouds ComputeService API is inefficient and very slow.

The following (scala) code is an example of a slow call:
{noformat}
val context = ContextBuilder.newBuilder("openstack-nova")
    .endpoint(origin.endpoint)
    .credentials(s"${origin.tenant}:${origin.user}", origin.secret)
    .build(classOf[ComputeServiceContext])
  val compute = context.getComputeService
  val securityGroupExtension = compute.getSecurityGroupExtension.get
  val secGroups = securityGroupExtension.listSecurityGroups
  val result = secGroups.map ( fromJCloud(_, existingGroups) )
{noformat}

Most of this is very quick, but when debugging I the call to listSecurityGroups is slow. In one example run this took 6 minutes and 17 seconds to retrieve 32 security groups. During this example call, 93 separate requests are made to the Openstack API server. The first is a call to GET /v2/<id>/extensions and the following 92 are calls to GET /v2/<id>/os-security-groups. These 92 calls all get an identical response from the API and the calls are made with about 4 seconds delay. After all these calls the method returns with the correct response.

One of the causes (but this doesn't seem to account for all the the excess calls I'm seeing) is that the NovaSecurityGroupToSecurityGroup class makes calls to transform each openstack SecurityGroupRule into a JClouds IpPermission object. This is done in the SecurityGroupRuleToIpPermission class, which takes a Predicate<AtomicReference<ZoneAndName>> and the supplied predicate is called for every transformation that occurs in which the source is another security group instead of a CIDR. The implementation of the predicate is FindSecurityGroupWithNameAndReturnTrue - which makes an API request for the list of security groups in a zone and then returns true if the supplied reference is in the newly obtained list. The API call is made regardless of whether it is the same zone or not.

In the particular zone I'm querying, there are a large number of rules (60) that reference other security groups (all in the same zone). As I said, this doesn't account for all the calls for exactly the same request - but it does account for more than half of them. I think there must be another transformation that also uses the predicate. 

I note that the predicate actually modifies the object, which makes it trickier to remove without changing the behaviour.

As a work around I am now using the NovaApi directly and can report that it takes a few seconds, instead of the 6 minutes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)