You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@libcloud.apache.org by GitBox <gi...@apache.org> on 2021/03/20 16:10:10 UTC

[GitHub] [libcloud] c-w opened a new pull request #1572: Add integration tests for Azure Storage driver

c-w opened a new pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572


   ## Add integration tests for Azure Storage driver
   
   ### Description
   
   As discussed in https://github.com/apache/libcloud/pull/1553#discussion_r572431845, this pull request ports the Azure Storage driver integration tests from [libcloud-tests](https://github.com/c-w/libcloud-tests) to trunk.
   
   The tests are split into a generic storage integration test base (which uses the high level storage driver API to run various scenarios) and specific instantiations which exercise various Azure Storage backends. Note that while libcloud-tests ran against containerized and cloud-based storage accounts, for now this pull request only ports the former as I will soon loose access to my free cloud credits.
   
   ### Status
   
   - done, ready for review
   
   ### Checklist (tick everything that applies)
   
   - [ ] [Code linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
   - [x] Documentation **n/a: new tests**
   - [x] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)
   - [x] [ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] c-w commented on a change in pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
c-w commented on a change in pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#discussion_r599610000



##########
File path: integration/storage/base.py
##########
@@ -0,0 +1,359 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import gzip
+import io
+import os
+import random
+import re
+import socket
+import string
+import tempfile
+import time
+import unittest
+
+import requests
+
+try:
+    import docker
+except ImportError:
+    docker = None
+
+from libcloud.storage import providers, types
+
+
+MB = 1024 * 1024
+
+
+class Integration:
+    class TestBase(unittest.TestCase):
+        provider = None
+        account = None
+        secret = None
+
+        def setUp(self):
+            for required in 'provider', 'account', 'secret':
+                value = getattr(self, required, None)
+                if value is None:
+                    raise unittest.SkipTest('config {} not set'.format(required))
+
+            kwargs = {'key': self.account, 'secret': self.secret}
+
+            for optional in 'host', 'port', 'secure':
+                value = getattr(self, optional, None)
+                if value is not None:
+                    kwargs[optional] = value
+
+            driver_class = providers.get_driver(self.provider)
+            self.driver = driver_class(**kwargs)
+
+        def tearDown(self):
+            for container in self.driver.list_containers():

Review comment:
       Done in 0c4d0937.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] Kami commented on a change in pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
Kami commented on a change in pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#discussion_r598294275



##########
File path: integration/storage/base.py
##########
@@ -0,0 +1,359 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import gzip
+import io
+import os
+import random
+import re
+import socket
+import string
+import tempfile
+import time
+import unittest
+
+import requests
+
+try:
+    import docker
+except ImportError:
+    docker = None
+
+from libcloud.storage import providers, types
+
+
+MB = 1024 * 1024
+
+
+class Integration:
+    class TestBase(unittest.TestCase):
+        provider = None
+        account = None
+        secret = None
+
+        def setUp(self):
+            for required in 'provider', 'account', 'secret':
+                value = getattr(self, required, None)
+                if value is None:
+                    raise unittest.SkipTest('config {} not set'.format(required))
+
+            kwargs = {'key': self.account, 'secret': self.secret}
+
+            for optional in 'host', 'port', 'secure':
+                value = getattr(self, optional, None)
+                if value is not None:
+                    kwargs[optional] = value
+
+            driver_class = providers.get_driver(self.provider)
+            self.driver = driver_class(**kwargs)
+
+        def tearDown(self):
+            for container in self.driver.list_containers():

Review comment:
       Do we




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] codecov-commenter commented on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-830675975


   # [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1572](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2024981) into [trunk](https://codecov.io/gh/apache/libcloud/commit/cc5987b5f79d4bca9c9139f24878686224b75425?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cc5987b) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/libcloud/pull/1572/graphs/tree.svg?width=650&height=150&src=pr&token=PYoduksh69&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@           Coverage Diff           @@
   ##            trunk    #1572   +/-   ##
   =======================================
     Coverage   83.00%   83.00%           
   =======================================
     Files         394      394           
     Lines       84990    84990           
     Branches     9038     9038           
   =======================================
     Hits        70550    70550           
     Misses      11373    11373           
     Partials     3067     3067           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [cc5987b...2024981](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] codecov-io edited a comment on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-803509002


   # [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=h1) Report
   > Merging [#1572](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=desc) (6803023) into [trunk](https://codecov.io/gh/apache/libcloud/commit/cc5987b5f79d4bca9c9139f24878686224b75425?el=desc) (cc5987b) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/libcloud/pull/1572/graphs/tree.svg?width=650&height=150&src=pr&token=PYoduksh69)](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##            trunk    #1572   +/-   ##
   =======================================
     Coverage   83.00%   83.00%           
   =======================================
     Files         394      394           
     Lines       84990    84990           
     Branches     9038     9038           
   =======================================
     Hits        70550    70550           
     Misses      11373    11373           
     Partials     3067     3067           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=footer). Last update [cc5987b...6803023](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] codecov-commenter edited a comment on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-830675975


   # [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#1572](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2024981) into [trunk](https://codecov.io/gh/apache/libcloud/commit/cc5987b5f79d4bca9c9139f24878686224b75425?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cc5987b) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/libcloud/pull/1572/graphs/tree.svg?width=650&height=150&src=pr&token=PYoduksh69&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@           Coverage Diff           @@
   ##            trunk    #1572   +/-   ##
   =======================================
     Coverage   83.00%   83.00%           
   =======================================
     Files         394      394           
     Lines       84990    84990           
     Branches     9038     9038           
   =======================================
     Hits        70550    70550           
     Misses      11373    11373           
     Partials     3067     3067           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [cc5987b...2024981](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] codecov-io edited a comment on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-803509002


   # [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=h1) Report
   > Merging [#1572](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=desc) (58cd26c) into [trunk](https://codecov.io/gh/apache/libcloud/commit/cc5987b5f79d4bca9c9139f24878686224b75425?el=desc) (cc5987b) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   > :exclamation: Current head 58cd26c differs from pull request most recent head e0f3026. Consider uploading reports for the commit e0f3026 to get more accurate results
   [![Impacted file tree graph](https://codecov.io/gh/apache/libcloud/pull/1572/graphs/tree.svg?width=650&height=150&src=pr&token=PYoduksh69)](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##            trunk    #1572   +/-   ##
   =======================================
     Coverage   83.00%   83.00%           
   =======================================
     Files         394      394           
     Lines       84990    84990           
     Branches     9038     9038           
   =======================================
     Hits        70550    70550           
     Misses      11373    11373           
     Partials     3067     3067           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=footer). Last update [cc5987b...e0f3026](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] codecov-io commented on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-803509002


   # [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=h1) Report
   > Merging [#1572](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=desc) (e24f58e) into [trunk](https://codecov.io/gh/apache/libcloud/commit/cc5987b5f79d4bca9c9139f24878686224b75425?el=desc) (cc5987b) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/libcloud/pull/1572/graphs/tree.svg?width=650&height=150&src=pr&token=PYoduksh69)](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##            trunk    #1572   +/-   ##
   =======================================
     Coverage   83.00%   83.00%           
   =======================================
     Files         394      394           
     Lines       84990    84990           
     Branches     9038     9038           
   =======================================
     Hits        70550    70550           
     Misses      11373    11373           
     Partials     3067     3067           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=footer). Last update [cc5987b...e24f58e](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] c-w merged pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
c-w merged pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] Kami commented on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
Kami commented on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-803602068


   Thanks for working on this.
   
   > While libcloud-tests ran against containerized and cloud-based storage accounts, for now this pull request only ports the former as I will soon loose access to my free cloud credits. However, if someone wants to add me to an Azure subscription that is funded for this project, or provide me with the credentials for a service principal, I'm more than happy to add the required code to run the test suite against the live cloud storage backends.
   
   @tonybaloney Do you happen to know if Azure has some kind of free credits for OSS program, similar to the AWS one which we could use for running end to end tests?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] Kami commented on a change in pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
Kami commented on a change in pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#discussion_r613887806



##########
File path: tox.ini
##########
@@ -212,10 +212,15 @@ commands =
     bash ./scripts/check_file_names.sh
     python ./scripts/check_asf_license_headers.py .
 
-[testenv:integration]
-deps = -r{toxinidir}/integration/requirements.txt
+[testenv:integration-compute]
+deps = -r{toxinidir}/integration/compute/requirements.txt
 
-commands = python -m integration
+commands = python -m integration.compute
+
+[testenv:integration-storage]
+deps = -r{toxinidir}/integration/storage/requirements.txt
+
+commands = python -m integration.storage

Review comment:
       This is fine for now, although in the future I would prefer to use ``pytest`` directly - this way common pytest flags (e.g. ``-s``) and plugins can be used.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] Kami commented on a change in pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
Kami commented on a change in pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#discussion_r613889227



##########
File path: tox.ini
##########
@@ -212,10 +212,15 @@ commands =
     bash ./scripts/check_file_names.sh
     python ./scripts/check_asf_license_headers.py .
 
-[testenv:integration]
-deps = -r{toxinidir}/integration/requirements.txt
+[testenv:integration-compute]
+deps = -r{toxinidir}/integration/compute/requirements.txt
 
-commands = python -m integration
+commands = python -m integration.compute
+
+[testenv:integration-storage]
+deps = -r{toxinidir}/integration/storage/requirements.txt
+
+commands = python -m integration.storage

Review comment:
       EDIT: Sorry, I just see you followed an existing pattern since original compute integration tests code is quite old :)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] Kami commented on a change in pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
Kami commented on a change in pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#discussion_r598294275



##########
File path: integration/storage/base.py
##########
@@ -0,0 +1,359 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import gzip
+import io
+import os
+import random
+import re
+import socket
+import string
+import tempfile
+import time
+import unittest
+
+import requests
+
+try:
+    import docker
+except ImportError:
+    docker = None
+
+from libcloud.storage import providers, types
+
+
+MB = 1024 * 1024
+
+
+class Integration:
+    class TestBase(unittest.TestCase):
+        provider = None
+        account = None
+        secret = None
+
+        def setUp(self):
+            for required in 'provider', 'account', 'secret':
+                value = getattr(self, required, None)
+                if value is None:
+                    raise unittest.SkipTest('config {} not set'.format(required))
+
+            kwargs = {'key': self.account, 'secret': self.secret}
+
+            for optional in 'host', 'port', 'secure':
+                value = getattr(self, optional, None)
+                if value is not None:
+                    kwargs[optional] = value
+
+            driver_class = providers.get_driver(self.provider)
+            self.driver = driver_class(**kwargs)
+
+        def tearDown(self):
+            for container in self.driver.list_containers():

Review comment:
       Should we wrap this with try / except and still proceed if there is an error (but I guess we should at least log an error if there is an exception and instruct user to manually delete those containers or similar).




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] codecov-io edited a comment on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-803509002


   # [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=h1) Report
   > Merging [#1572](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=desc) (e0f3026) into [trunk](https://codecov.io/gh/apache/libcloud/commit/cc5987b5f79d4bca9c9139f24878686224b75425?el=desc) (cc5987b) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/libcloud/pull/1572/graphs/tree.svg?width=650&height=150&src=pr&token=PYoduksh69)](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##            trunk    #1572   +/-   ##
   =======================================
     Coverage   83.00%   83.00%           
   =======================================
     Files         394      394           
     Lines       84990    84990           
     Branches     9038     9038           
   =======================================
     Hits        70550    70550           
     Misses      11373    11373           
     Partials     3067     3067           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=footer). Last update [cc5987b...e0f3026](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] tonybaloney commented on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
tonybaloney commented on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-805260481


   I've put in an internal request at Microsoft for some credit.
   
   I'll ping back on here when/if it gets approved
   
   On Wed, Mar 24, 2021 at 1:18 AM Clemens Wolff ***@***.***>
   wrote:
   
   > ***@***.**** commented on this pull request.
   > ------------------------------
   >
   > In integration/storage/base.py
   > <https://github.com/apache/libcloud/pull/1572#discussion_r599610000>:
   >
   > > +                value = getattr(self, required, None)
   > +                if value is None:
   > +                    raise unittest.SkipTest('config {} not set'.format(required))
   > +
   > +            kwargs = {'key': self.account, 'secret': self.secret}
   > +
   > +            for optional in 'host', 'port', 'secure':
   > +                value = getattr(self, optional, None)
   > +                if value is not None:
   > +                    kwargs[optional] = value
   > +
   > +            driver_class = providers.get_driver(self.provider)
   > +            self.driver = driver_class(**kwargs)
   > +
   > +        def tearDown(self):
   > +            for container in self.driver.list_containers():
   >
   > Done in 0c4d093
   > <https://github.com/apache/libcloud/commit/0c4d09376c824eb542743fc225fc30c22ab39bc9>
   > .
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/libcloud/pull/1572#discussion_r599610000>, or
   > unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/AALWEAJV7Q7DO3ZMT5IMGXTTFCPKHANCNFSM4ZQSB2FA>
   > .
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] c-w commented on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
c-w commented on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-803717907


   @Kami @tonybaloney Given that ASF is a non-profit, the [Azure for non-profits grant](https://www.microsoft.com/en-us/nonprofits/azure) could be an avenue to get some credits.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] c-w commented on a change in pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
c-w commented on a change in pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#discussion_r599610000



##########
File path: integration/storage/base.py
##########
@@ -0,0 +1,359 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import gzip
+import io
+import os
+import random
+import re
+import socket
+import string
+import tempfile
+import time
+import unittest
+
+import requests
+
+try:
+    import docker
+except ImportError:
+    docker = None
+
+from libcloud.storage import providers, types
+
+
+MB = 1024 * 1024
+
+
+class Integration:
+    class TestBase(unittest.TestCase):
+        provider = None
+        account = None
+        secret = None
+
+        def setUp(self):
+            for required in 'provider', 'account', 'secret':
+                value = getattr(self, required, None)
+                if value is None:
+                    raise unittest.SkipTest('config {} not set'.format(required))
+
+            kwargs = {'key': self.account, 'secret': self.secret}
+
+            for optional in 'host', 'port', 'secure':
+                value = getattr(self, optional, None)
+                if value is not None:
+                    kwargs[optional] = value
+
+            driver_class = providers.get_driver(self.provider)
+            self.driver = driver_class(**kwargs)
+
+        def tearDown(self):
+            for container in self.driver.list_containers():

Review comment:
       Done in d9f577e8.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] codecov-io edited a comment on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-803509002


   # [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=h1) Report
   > Merging [#1572](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=desc) (f0872ff) into [trunk](https://codecov.io/gh/apache/libcloud/commit/cc5987b5f79d4bca9c9139f24878686224b75425?el=desc) (cc5987b) will **not change** coverage.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/libcloud/pull/1572/graphs/tree.svg?width=650&height=150&src=pr&token=PYoduksh69)](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##            trunk    #1572   +/-   ##
   =======================================
     Coverage   83.00%   83.00%           
   =======================================
     Files         394      394           
     Lines       84990    84990           
     Branches     9038     9038           
   =======================================
     Hits        70550    70550           
     Misses      11373    11373           
     Partials     3067     3067           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=footer). Last update [cc5987b...f0872ff](https://codecov.io/gh/apache/libcloud/pull/1572?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [libcloud] c-w commented on pull request #1572: Add integration tests for Azure Storage driver

Posted by GitBox <gi...@apache.org>.
c-w commented on pull request #1572:
URL: https://github.com/apache/libcloud/pull/1572#issuecomment-818706254


   @tonybaloney @Kami Any thoughts on next steps for this integration test PR?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org