[jira] [Created] (AIRFLOW-2705) Move class-level moto decorator to method-level

["Kengo Seki (JIRA)" <ji...@apache.org>]

Kengo Seki created AIRFLOW-2705:
-----------------------------------

             Summary: Move class-level moto decorator to method-level
                 Key: AIRFLOW-2705
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2705
             Project: Apache Airflow
          Issue Type: Bug
          Components: redshift, tests
            Reporter: Kengo Seki
            Assignee: Kengo Seki


In tests/contrib/hooks/test_redshift_hook.py and tests/contrib/sensors/test_aws_redshift_cluster_sensor.py, moto decorators are at class-level, whereas they are at method-level in other tests using moto.

{code:title=tests/contrib/hooks/test_redshift_hook.py}
@mock_redshift
class TestRedshiftHook(unittest.TestCase):
{code}

I found it can affect other tests using HTTP request. Let's say the following test is added to tests/hooks/test_http_hook.py:

{code:title=tests/hooks/test_http_hook.py}
    def test_sample(self):
        HttpHook("GET").run("")
{code}

This test sends a GET request to https://www.google.com/ and succeeds of course:

{code}
$ ./run_unit_tests.sh tests.hooks.test_http_hook:TestHttpHook.test_sample

(snip)

/home/sekikn/a3/lib/python3.5/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
.
----------------------------------------------------------------------
Ran 1 test in 0.189s

OK
{code}

But strangely, it fails with ConnectionError when it's executed after tests/contrib/hooks/test_redshift_hook.py or tests/contrib/sensors/test_aws_redshift_cluster_sensor.py:

{code}
$ ./run_unit_tests.sh tests.contrib.hooks.test_redshift_hook tests.hooks.test_http_hook:TestHttpHook.test_sample

(snip)

E
======================================================================
ERROR: test_sample (tests.hooks.test_http_hook.TestHttpHook)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sekikn/dev/incubator-airflow/tests/hooks/test_http_hook.py", line 250, in test_sample
    HttpHook("GET").run("")
  File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line 119, in run
    return self.run_and_check(session, prepped_request, extra_options)
  File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line 167, in run_and_check
    raise ex
  File "/home/sekikn/dev/incubator-airflow/airflow/hooks/http_hook.py", line 159, in run_and_check
    allow_redirects=extra_options.get("allow_redirects", True))
  File "/home/sekikn/a3/lib/python3.5/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/home/sekikn/a3/lib/python3.5/site-packages/moto/packages/responses/responses.py", line 308, in unbound_on_send
    return self._on_request(adapter, request, *a, **kwargs)
  File "/home/sekikn/a3/lib/python3.5/site-packages/moto/packages/responses/responses.py", line 250, in _on_request
    raise response
requests.exceptions.ConnectionError: Connection refused: GET https://www.google.com/
-------------------- >> begin captured stdout << ---------------------
[2018-06-30 09:41:21,029] {base_hook.py:83} INFO - Using connection to: https://www.google.com/
[2018-06-30 09:41:21,030] {http_hook.py:118} INFO - Sending 'GET' to url: https://www.google.com/
[2018-06-30 09:41:21,030] {http_hook.py:166} WARNING - Connection refused: GET https://www.google.com/ Tenacity will retry to execute the operation

--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
airflow.utils.log.logging_mixin.LoggingMixin: INFO: Using connection to: https://www.google.com/
airflow.hooks.http_hook.HttpHook: INFO: Sending 'GET' to url: https://www.google.com/
airflow.hooks.http_hook.HttpHook: WARNING: Connection refused: GET https://www.google.com/ Tenacity will retry to execute the operation
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 7 tests in 1.838s

FAILED (errors=1)
{code}

This problem is solved by using method-level moto decorator instead of class-level one. I suspect class-level decorator patches method that sends HTTP request and doesn't restore the original state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)