[GitHub] libcloud pull request #1203: Fixed bug where the method _find_zone_or_region...

[Ahuri3 <gi...@git.apache.org>]

GitHub user Ahuri3 opened a pull request:

    https://github.com/apache/libcloud/pull/1203

    Fixed bug where the method _find_zone_or_region cannot see some instances

    
    ## Fixed bug where _find_zone_or_region cannot see some instances
    
    ### Description
    
    The method `_find_zone_or_region` queries the google cloud api on the endpoint `instances.aggregatedList`.  
    
    This endpoint cannot return more than 500 results but has a pagination mechanic. The method `_find_zone_or_region` does not make use of the pagination mechanic because it made a manual request to the api.
    
    I removed the manual request and replaced it by the  method `request_aggregated_items` that was made for this exact purpose.
    
    This bug was found while trying to use the libcloud in ansible (via the gce_snapshot ansible module) and getting an error `libcloud.common.google.ResourceNotFoundError: "Node 'node_name' not found in any zone."`. 
    
    Indeed without the pagination, if a google cloud project has more than 500 instances, the code will crash when trying to identify the zone for a gce instance that wasn't in the 500 instances returned by the first api call. 
    
    Here is the stack trace I saw : 
    ```
    The full traceback is:
    Traceback (most recent call last):
      File "/tmp/ansible_qBlsq5/ansible_module_gce_snapshot.py", line 225, in <module>
        main()
      File "/tmp/ansible_qBlsq5/ansible_module_gce_snapshot.py", line 190, in main
        instance = gce.ex_get_node(instance_name, 'all')
      File "/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py", line 7030, in ex_get_node
        name, 'instances', res_name='Node')
      File "/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py", line 7561, in _find_zone_or_region
        (res_name, name, rz), None, None)
    libcloud.common.google.ResourceNotFoundError: "Node 'pr-op-cass00-0004' not found in any zone."
    fatal: [pr-op-cass00-0004 -> localhost]: FAILED! => {
        "changed": false, 
        "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_Gk7vXV/ansible_module_gce_snapshot.py\", line 225, in <module>\n    main()\n  File \"/tmp/ansible_Gk7vXV/ansible_module_gce_snapshot.py\", line 190, in main\n    instance = gce.ex_get_node(instance_name, 'all')\n  File \"/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py\", line 7030, in ex_get_node\n    name, 'instances', res_name='Node')\n  File \"/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py\", line 7561, in _find_zone_or_region\n    (res_name, name, rz), None, None)\nlibcloud.common.google.ResourceNotFoundError: \"Node 'pr-op-cass00-0004' not found in any zone.\"\n", 
        "module_stdout": "", 
        "msg": "MODULE FAILURE", 
        "rc": 1
    }
    ...ignoring
    ```
    
    ### Status
    
    ready for review
    
    ### Checklist (tick everything that applies)
    
    - [x] [Code linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide) (required, can be done after the PR checks)
      - No pagination was changed
      - Line lenght is shorter than 79 characters
    - [ ] Documentation
      - N/A
    - [x] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)
      - Tested with `Python 2.7.13 :: Continuum Analytics, Inc.`
    - [ ] [ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes) (required for bigger changes)
      - N/A


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/Ahuri3/libcloud fix_find_zone_or_region_by_using_request_aggregated_items

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/libcloud/pull/1203.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1203
    
----
commit 4a7d884ca6e9d1864320bf3b21087fb66ac21d01
Author: Léo Ferlin-Sutton <lf...@...>
Date:   2018-04-26T13:22:17Z

    Modified the way '_find_zone_or_region' gets data from the 'instances.aggregatedList' api by using the method 'request_aggregated_items' which correctly paginates

----


---

[GitHub] libcloud pull request #1203: Fixed bug where the method _find_zone_or_region...

[asfgit <gi...@git.apache.org>]

Github user asfgit closed the pull request at:

    https://github.com/apache/libcloud/pull/1203


---