You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@libcloud.apache.org by to...@apache.org on 2021/11/07 18:42:40 UTC
[libcloud] branch trunk updated (748dc59 -> b0eeb3f)
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git.
from 748dc59 Add changelog entry for #1610.
new 2d1ad6a Run unit tests in parallel to speed up the test run.
new edc80a3 Use longer sleep to make failure less likely.
new 1bd52a7 Fix bug in the test which was causing a test failure under some scenarios when running tests in standalone fashion (so compute test didn't set token value on which test relies on).
new c275a93 Test a fix for Python 3.5 compatibility issue.
new 2d76c2c Fix Python 3.5 compatibility.
new 26d2782 Use longer timeout to make failure less likely.
new debbf9f Update requests mock, test a change.
new 20ae65f Raise sleep interval even more.
new 58d22eb Try a fix for test failure.
new ee9bbc4 Remove debug print.
new e3c2223 Remove dummy test case we don't need anymore.
new ecc5cc3 Add changelog entry.
new 8100a6e Merge branch 'trunk' into pytest_parallel
new b0eeb3f Update changelog.
The 14 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGES.rst | 4 ++++
libcloud/test/loadbalancer/test_brightbox.py | 1 +
libcloud/test/storage/test_local.py | 7 +++++--
libcloud/test/test_utils.py | 7 -------
requirements-tests.txt | 4 +++-
tox.ini | 8 +++++++-
6 files changed, 20 insertions(+), 11 deletions(-)
[libcloud] 11/14: Remove dummy test case we don't need anymore.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit e3c2223b4151c003056d58d9e8f9f6c3acbe1f69
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sun Nov 7 19:16:40 2021 +0100
Remove dummy test case we don't need anymore.
---
libcloud/test/test_utils.py | 7 -------
1 file changed, 7 deletions(-)
diff --git a/libcloud/test/test_utils.py b/libcloud/test/test_utils.py
index 05590e8..ed7109a 100644
--- a/libcloud/test/test_utils.py
+++ b/libcloud/test/test_utils.py
@@ -433,12 +433,5 @@ def test_decorator():
foo()
-def test_get_response_object():
- with requests_mock.mock() as m:
- m.get('https://test.com/test', text='data')
- response = get_response_object('https://test.com/test')
- assert response.body == 'data'
-
-
if __name__ == '__main__':
sys.exit(unittest.main())
[libcloud] 03/14: Fix bug in the test which was causing a test
failure under some scenarios when running tests in standalone fashion (so
compute test didn't set token value on which test relies on).
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit 1bd52a7019eb0ec47fb3594f2d67f2e36a02b823
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sat Nov 6 23:45:35 2021 +0100
Fix bug in the test which was causing a test failure under some
scenarios when running tests in standalone fashion (so compute test
didn't set token value on which test relies on).
---
libcloud/test/loadbalancer/test_brightbox.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/libcloud/test/loadbalancer/test_brightbox.py b/libcloud/test/loadbalancer/test_brightbox.py
index 93594b0..1617f77 100644
--- a/libcloud/test/loadbalancer/test_brightbox.py
+++ b/libcloud/test/loadbalancer/test_brightbox.py
@@ -30,6 +30,7 @@ class BrightboxLBTests(unittest.TestCase):
def setUp(self):
BrightboxLBDriver.connectionCls.conn_class = BrightboxLBMockHttp
BrightboxLBMockHttp.type = None
+ BrightboxLBDriver.connectionCls.token = 'test'
self.driver = BrightboxLBDriver(*LB_BRIGHTBOX_PARAMS)
def test_list_protocols(self):
[libcloud] 01/14: Run unit tests in parallel to speed up the test
run.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit 2d1ad6adaf9c0f2d0b6402b3f4c276dca484e3c3
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sat Nov 6 23:22:43 2021 +0100
Run unit tests in parallel to speed up the test run.
---
requirements-tests.txt | 2 ++
tox.ini | 6 +++++-
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/requirements-tests.txt b/requirements-tests.txt
index 118c425..5547fc9 100644
--- a/requirements-tests.txt
+++ b/requirements-tests.txt
@@ -15,6 +15,8 @@ requests_mock
# 5.3.2 is latest version which still supports Python 3.5, >= 6.2.5 is needed for Python 3.10
pytest==5.3.2; python_version <= '3.5'
pytest==6.2.5; python_version >= '3.6'
+pytest-xdist==2.2.1; python_version <= '3.5'
+pytest-xdist==2.4.0; python_version >= '3.6'
cryptography==3.2.1; python_version <= '3.5'
cryptography==35.0.0; python_version >= '3.6'
# NOTE: Only needed by nttcis loadbalancer driver
diff --git a/tox.ini b/tox.ini
index f5c3886..82c90d9 100644
--- a/tox.ini
+++ b/tox.ini
@@ -21,8 +21,12 @@ basepython =
{py3.10,py3.10-dist,py3.10-dist-wheel}: python3.10
setenv =
CRYPTOGRAPHY_ALLOW_OPENSSL_102=1
+# NOTE: By default we run tests on CI in parallel to speed up the build
+# To avoid per-test function process safety issues we run all tests in a single
+# file in the same worker process.
+# python setup.py test
commands = cp libcloud/test/secrets.py-dist libcloud/test/secrets.py
- python setup.py test
+ pytest -rsx -vvv --capture=tee-sys -o log_cli=True --durations=10 -n auto --dist loadfile
whitelist_externals = cp
bash
[libcloud] 04/14: Test a fix for Python 3.5 compatibility issue.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit c275a932b820e545b6759f1e20169096fefe84c0
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sat Nov 6 23:51:01 2021 +0100
Test a fix for Python 3.5 compatibility issue.
---
requirements-tests.txt | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/requirements-tests.txt b/requirements-tests.txt
index 5547fc9..5a9c6ca 100644
--- a/requirements-tests.txt
+++ b/requirements-tests.txt
@@ -15,7 +15,7 @@ requests_mock
# 5.3.2 is latest version which still supports Python 3.5, >= 6.2.5 is needed for Python 3.10
pytest==5.3.2; python_version <= '3.5'
pytest==6.2.5; python_version >= '3.6'
-pytest-xdist==2.2.1; python_version <= '3.5'
+pytest-xdist==1.34.0; python_version <= '3.5'
pytest-xdist==2.4.0; python_version >= '3.6'
cryptography==3.2.1; python_version <= '3.5'
cryptography==35.0.0; python_version >= '3.6'
[libcloud] 12/14: Add changelog entry.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit ecc5cc3aa8153860d6fb7bac98bef7e940bba38f
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sun Nov 7 19:18:27 2021 +0100
Add changelog entry.
---
CHANGES.rst | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/CHANGES.rst b/CHANGES.rst
index b69911d..6aa1c30 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,6 +1,16 @@
Changelog
=========
+Changes in Apache Libcloud in development
+-----------------------------------------
+
+Development
+~~~~~~~~~~~
+
+- Update tox targets for unit tests to utilize ``pytest-xdist`` plugin to run
+ tests in parallel in multiple processes to speed up the test runs.
+ (GITHUB-1625)
+
Changes in Apache Libcloud 3.4.0
--------------------------------
[libcloud] 05/14: Fix Python 3.5 compatibility.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit 2d76c2c888d7d068595c01f4ee24d96968195fb2
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sat Nov 6 23:56:40 2021 +0100
Fix Python 3.5 compatibility.
---
tox.ini | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/tox.ini b/tox.ini
index 82c90d9..d0ae930 100644
--- a/tox.ini
+++ b/tox.ini
@@ -25,8 +25,10 @@ setenv =
# To avoid per-test function process safety issues we run all tests in a single
# file in the same worker process.
# python setup.py test
+# NOTE: tee-sys is not supported by pytest which still supports Python 3.5
+# pytest -rsx -vvv --capture=tee-sys -o log_cli=True --durations=10 -n auto --dist loadfile
commands = cp libcloud/test/secrets.py-dist libcloud/test/secrets.py
- pytest -rsx -vvv --capture=tee-sys -o log_cli=True --durations=10 -n auto --dist loadfile
+ pytest -rsx -vvv -o log_cli=True --durations=10 -n auto --dist loadfile
whitelist_externals = cp
bash
[libcloud] 10/14: Remove debug print.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit ee9bbc4041a493d78edf31298e8042639401b8f8
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sun Nov 7 00:43:59 2021 +0100
Remove debug print.
---
libcloud/test/storage/test_local.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/libcloud/test/storage/test_local.py b/libcloud/test/storage/test_local.py
index 1a578cd..002265c 100644
--- a/libcloud/test/storage/test_local.py
+++ b/libcloud/test/storage/test_local.py
@@ -76,7 +76,6 @@ class LocalTests(unittest.TestCase):
@unittest.skipIf(platform.system().lower() == 'windows', 'Unsupported on Windows')
def test_lock_local_storage(self):
- print("aaaa")
# 1. Acquire succeeds
lock = LockLocalStorage("/tmp/a")
with lock:
[libcloud] 13/14: Merge branch 'trunk' into pytest_parallel
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit 8100a6e23f15907ee90a911ff1bd0b9ea7f7a00a
Merge: ecc5cc3 748dc59
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sun Nov 7 19:26:43 2021 +0100
Merge branch 'trunk' into pytest_parallel
CHANGES.rst | 14 +-
libcloud/common/vultr.py | 86 +-
libcloud/compute/drivers/vultr.py | 1049 +++++++++++++++++++-
libcloud/dns/drivers/vultr.py | 342 ++++++-
.../compute/fixtures/vultr_v2/create_node.json | 34 +
.../fixtures/vultr_v2/create_node_bare_metal.json | 27 +
.../compute/fixtures/vultr_v2/create_volume.json | 13 +
.../fixtures/vultr_v2/ex_create_network.json | 10 +
.../fixtures/vultr_v2/ex_create_snapshot.json | 11 +
.../compute/fixtures/vultr_v2/ex_get_network.json | 10 +
.../compute/fixtures/vultr_v2/ex_get_node.json | 31 +
.../compute/fixtures/vultr_v2/ex_get_snapshot.json | 11 +
.../compute/fixtures/vultr_v2/ex_get_volume.json | 13 +
.../ex_list_available_sizes_for_location.json | 21 +
.../vultr_v2/ex_list_bare_metal_nodes.json | 59 ++
.../vultr_v2/ex_list_bare_metal_sizes.json | 88 ++
.../fixtures/vultr_v2/ex_list_networks.json | 27 +
.../fixtures/vultr_v2/ex_list_snapshots.json | 29 +
.../compute/fixtures/vultr_v2/ex_resize_node.json | 31 +
.../compute/fixtures/vultr_v2/get_key_pair.json | 8 +
.../vultr_v2/import_key_pair_from_string.json | 8 +
.../compute/fixtures/vultr_v2/list_images.json | 191 ++++
.../fixtures/vultr_v2/list_images_paginated_1.json | 71 ++
.../fixtures/vultr_v2/list_images_paginated_2.json | 71 ++
.../fixtures/vultr_v2/list_images_paginated_3.json | 71 ++
.../compute/fixtures/vultr_v2/list_key_pairs.json | 17 +
.../compute/fixtures/vultr_v2/list_locations.json | 154 +++
.../test/compute/fixtures/vultr_v2/list_nodes.json | 98 ++
.../test/compute/fixtures/vultr_v2/list_sizes.json | 518 ++++++++++
.../compute/fixtures/vultr_v2/list_volumes.json | 33 +
libcloud/test/compute/test_vultr.py | 8 +-
libcloud/test/compute/test_vultr_v2.py | 554 +++++++++++
.../test/dns/fixtures/vultr_v2/create_record.json | 10 +
.../test/dns/fixtures/vultr_v2/create_zone.json | 6 +
.../test/dns/fixtures/vultr_v2/get_record.json | 10 +
libcloud/test/dns/fixtures/vultr_v2/get_zone.json | 6 +
.../test/dns/fixtures/vultr_v2/list_records.json | 51 +
.../test/dns/fixtures/vultr_v2/list_zones.json | 19 +
libcloud/test/dns/test_vultr.py | 9 +-
libcloud/test/dns/test_vultr_v2.py | 153 +++
40 files changed, 3954 insertions(+), 18 deletions(-)
[libcloud] 14/14: Update changelog.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit b0eeb3f9b8f5fbf8de6531873eea9830f8fff9ee
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sun Nov 7 19:32:14 2021 +0100
Update changelog.
---
CHANGES.rst | 14 ++++----------
1 file changed, 4 insertions(+), 10 deletions(-)
diff --git a/CHANGES.rst b/CHANGES.rst
index 8117566..74744fe 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,16 +1,6 @@
Changelog
=========
-Changes in Apache Libcloud in development
------------------------------------------
-
-Development
-~~~~~~~~~~~
-
-- Update tox targets for unit tests to utilize ``pytest-xdist`` plugin to run
- tests in parallel in multiple processes to speed up the test runs.
- (GITHUB-1625)
-
Changes in Apache Libcloud 3.4.0
--------------------------------
@@ -207,6 +197,10 @@ Other
(GITHUB-1606)
[Michał Górny - @mgorny]
+- Update tox targets for unit tests to utilize ``pytest-xdist`` plugin to run
+ tests in parallel in multiple processes to speed up the test runs.
+ (GITHUB-1625)
+
Changes in Apache Libcloud 3.3.1
--------------------------------
[libcloud] 07/14: Update requests mock, test a change.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit debbf9f76c8642c56299fbcbb6db42b7d1dae4ff
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sun Nov 7 00:03:33 2021 +0100
Update requests mock, test a change.
---
libcloud/test/test_utils.py | 4 ++--
requirements-tests.txt | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/libcloud/test/test_utils.py b/libcloud/test/test_utils.py
index cbec7bd..05590e8 100644
--- a/libcloud/test/test_utils.py
+++ b/libcloud/test/test_utils.py
@@ -435,8 +435,8 @@ def test_decorator():
def test_get_response_object():
with requests_mock.mock() as m:
- m.get('http://test.com/test', text='data')
- response = get_response_object('http://test.com/test')
+ m.get('https://test.com/test', text='data')
+ response = get_response_object('https://test.com/test')
assert response.body == 'data'
diff --git a/requirements-tests.txt b/requirements-tests.txt
index 5a9c6ca..09b3c18 100644
--- a/requirements-tests.txt
+++ b/requirements-tests.txt
@@ -11,7 +11,7 @@ codecov==2.1.12
coverage==4.5.4
requests>=2.25.0; python_version <= '3.5'
requests>=2.26.0; python_version >= '3.6'
-requests_mock
+requests_mock==1.9.3
# 5.3.2 is latest version which still supports Python 3.5, >= 6.2.5 is needed for Python 3.10
pytest==5.3.2; python_version <= '3.5'
pytest==6.2.5; python_version >= '3.6'
[libcloud] 09/14: Try a fix for test failure.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit 58d22eb300204c503ea5d2fb7cd82535da9353cf
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sun Nov 7 00:11:54 2021 +0100
Try a fix for test failure.
---
libcloud/test/storage/test_local.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/libcloud/test/storage/test_local.py b/libcloud/test/storage/test_local.py
index e798112..1a578cd 100644
--- a/libcloud/test/storage/test_local.py
+++ b/libcloud/test/storage/test_local.py
@@ -91,13 +91,13 @@ class LocalTests(unittest.TestCase):
# 3. Multiprocessing scenario where IPC lock is involved
def acquire_lock_in_subprocess(pid, success):
# For first process acquire should succeed and for the second it should fail
- lock = LockLocalStorage("/tmp/c", timeout=2.5)
+ lock = LockLocalStorage("/tmp/c", timeout=0.5)
if pid == 1:
with lock:
# We use longer sleep when running tests in parallel to avoid
# failures related to slower process spawn
- time.sleep(3)
+ time.sleep(2.5)
success.value = 1
elif pid == 2:
@@ -113,6 +113,8 @@ class LocalTests(unittest.TestCase):
p1 = multiprocessing.Process(target=acquire_lock_in_subprocess, args=(1, success_1,))
p1.start()
+ time.sleep(0.2)
+
p2 = multiprocessing.Process(target=acquire_lock_in_subprocess, args=(2, success_2,))
p2.start()
[libcloud] 06/14: Use longer timeout to make failure less likely.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit 26d27829f5e9d9a62ce1f27aa506375747d921d9
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sat Nov 6 23:59:32 2021 +0100
Use longer timeout to make failure less likely.
---
libcloud/test/storage/test_local.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/libcloud/test/storage/test_local.py b/libcloud/test/storage/test_local.py
index 3c579bd..0613f16 100644
--- a/libcloud/test/storage/test_local.py
+++ b/libcloud/test/storage/test_local.py
@@ -76,6 +76,7 @@ class LocalTests(unittest.TestCase):
@unittest.skipIf(platform.system().lower() == 'windows', 'Unsupported on Windows')
def test_lock_local_storage(self):
+ print("aaaa")
# 1. Acquire succeeds
lock = LockLocalStorage("/tmp/a")
with lock:
@@ -90,11 +91,12 @@ class LocalTests(unittest.TestCase):
# 3. Multiprocessing scenario where IPC lock is involved
def acquire_lock_in_subprocess(pid, success):
# For first process acquire should succeed and for the second it should fail
-
- lock = LockLocalStorage("/tmp/c", timeout=0.5)
+ lock = LockLocalStorage("/tmp/c", timeout=1.5)
if pid == 1:
with lock:
+ # We use longer sleep when running tests in parallel to avoid
+ # failures related to slower process spawn
time.sleep(2)
success.value = 1
[libcloud] 08/14: Raise sleep interval even more.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit 20ae65fe2015f019cca52da8247ff1f3610c24e8
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sun Nov 7 00:05:21 2021 +0100
Raise sleep interval even more.
---
libcloud/test/storage/test_local.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libcloud/test/storage/test_local.py b/libcloud/test/storage/test_local.py
index 0613f16..e798112 100644
--- a/libcloud/test/storage/test_local.py
+++ b/libcloud/test/storage/test_local.py
@@ -91,13 +91,13 @@ class LocalTests(unittest.TestCase):
# 3. Multiprocessing scenario where IPC lock is involved
def acquire_lock_in_subprocess(pid, success):
# For first process acquire should succeed and for the second it should fail
- lock = LockLocalStorage("/tmp/c", timeout=1.5)
+ lock = LockLocalStorage("/tmp/c", timeout=2.5)
if pid == 1:
with lock:
# We use longer sleep when running tests in parallel to avoid
# failures related to slower process spawn
- time.sleep(2)
+ time.sleep(3)
success.value = 1
elif pid == 2:
[libcloud] 02/14: Use longer sleep to make failure less likely.
Posted by to...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tomaz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/libcloud.git
commit edc80a33dfe5d8d40e83c4528cdc9d016bfd1aaf
Author: Tomaz Muraus <to...@tomaz.me>
AuthorDate: Sat Nov 6 23:44:29 2021 +0100
Use longer sleep to make failure less likely.
---
libcloud/test/storage/test_local.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/libcloud/test/storage/test_local.py b/libcloud/test/storage/test_local.py
index cfd5d44..3c579bd 100644
--- a/libcloud/test/storage/test_local.py
+++ b/libcloud/test/storage/test_local.py
@@ -95,7 +95,7 @@ class LocalTests(unittest.TestCase):
if pid == 1:
with lock:
- time.sleep(1)
+ time.sleep(2)
success.value = 1
elif pid == 2: