[jira] [Updated] (AIRFLOW-2705) Move class-level moto decorator to method-level

["Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aizhamal Nurmamat kyzy updated AIRFLOW-2705:
--------------------------------------------
         Labels: redshift  (was: )
    Component/s: aws

Moving issues to 'aws', adding 'redshift' label.

> Move class-level moto decorator to method-level
> -----------------------------------------------
>
>                 Key: AIRFLOW-2705
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2705
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: aws, redshift, tests
>            Reporter: Kengo Seki
>            Assignee: Kengo Seki
>            Priority: Major
>              Labels: redshift
>             Fix For: 1.10.2
>
>
> In tests/contrib/hooks/test_redshift_hook.py and tests/contrib/sensors/test_aws_redshift_cluster_sensor.py, moto decorators are at class-level, whereas they are at method-level in other tests using moto.
> {code:title=tests/contrib/hooks/test_redshift_hook.py}
> @mock_redshift
> class TestRedshiftHook(unittest.TestCase):
> {code}
> I found it can affect other tests using HTTP request. Let's say the following test is added to tests/hooks/test_http_hook.py:
> {code:title=tests/hooks/test_http_hook.py}
>     def test_sample(self):
>         HttpHook("GET").run("")
> {code}
> This test sends a GET request to https://www.google.com/ and succeeds of course:
> {code}
> $ ./run_unit_tests.sh tests.hooks.test_http_hook:TestHttpHook.test_sample
> (snip)
> /home/sekikn/a3/lib/python3.5/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
>   InsecureRequestWarning)
> .
> ----------------------------------------------------------------------
> Ran 1 test in 0.189s
> OK
> {code}
> But strangely, it fails with ConnectionError when it's executed after tests/contrib/hooks/test_redshift_hook.py or tests/contrib/sensors/test_aws_redshift_cluster_sensor.py:
> {code}
> $ ./run_unit_tests.sh tests.contrib.hooks.test_redshift_hook tests.hooks.test_http_hook:TestHttpHook.test_sample
> (snip)
> E
> ======================================================================
> ERROR: test_sample (tests.hooks.test_http_hook.TestHttpHook)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/sekikn/dev/incubator-airflow/tests/hooks/test_http_hook.py", line 250, in test_sample
>     HttpHook("GET").run("")
>   File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line 119, in run
>     return self.run_and_check(session, prepped_request, extra_options)
>   File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line 167, in run_and_check
>     raise ex
>   File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line 159, in run_and_check
>     allow_redirects=extra_options.get("allow_redirects", True))
>   File "/home/sekikn/a3/lib/python3.5/site-packages/requests/sessions.py", line 618, in send
>     r = adapter.send(request, **kwargs)
>   File "/home/sekikn/a3/lib/python3.5/site-packages/moto/packages/responses/responses.py", line 308, in unbound_on_send
>     return self._on_request(adapter, request, *a, **kwargs)
>   File "/home/sekikn/a3/lib/python3.5/site-packages/moto/packages/responses/responses.py", line 250, in _on_request
>     raise response
> requests.exceptions.ConnectionError: Connection refused: GET https://www.google.com/
> -------------------- >> begin captured stdout << ---------------------
> [2018-06-30 09:41:21,029] {base_hook.py:83} INFO - Using connection to: https://www.google.com/
> [2018-06-30 09:41:21,030] {http_hook.py:118} INFO - Sending 'GET' to url: https://www.google.com/
> [2018-06-30 09:41:21,030] {http_hook.py:166} WARNING - Connection refused: GET https://www.google.com/ Tenacity will retry to execute the operation
> --------------------- >> end captured stdout << ----------------------
> -------------------- >> begin captured logging << --------------------
> airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: https://www.google.com/
> airflow.hooks.http_hook.HttpHook: INFO: Sending 'GET' to url: https://www.google.com/
> airflow.hooks.http_hook.HttpHook: WARNING: Connection refused: GET https://www.google.com/ Tenacity will retry to execute the operation
> --------------------- >> end captured logging << ---------------------
> ----------------------------------------------------------------------
> Ran 7 tests in 1.838s
> FAILED (errors=1)
> {code}
> This problem is solved by using method-level moto decorator instead of class-level one. I suspect class-level decorator patches method that sends HTTP request and doesn't restore the original state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)