[GitHub] [airflow] potiuk commented on a change in pull request #22017: Removes limitations from Dask dependencies

[GitBox <gi...@apache.org>]

potiuk commented on a change in pull request #22017:
URL: https://github.com/apache/airflow/pull/22017#discussion_r820154297



##########
File path: tests/executors/test_dask_executor.py
##########
@@ -20,31 +20,39 @@
 from unittest import mock
 
 import pytest
+from distributed import LocalCluster
 
 from airflow.exceptions import AirflowException
+from airflow.executors.dask_executor import DaskExecutor
 from airflow.jobs.backfill_job import BackfillJob
 from airflow.models import DagBag
 from airflow.utils import timezone
 from tests.test_utils.config import conf_vars
 
 try:
-    from distributed import LocalCluster
-
     # utility functions imported from the dask testing suite to instantiate a test
     # cluster for tls tests
+    from distributed import tests  # noqa
     from distributed.utils_test import cluster as dask_testing_cluster, get_cert, tls_security
 
-    from airflow.executors.dask_executor import DaskExecutor
-
     skip_tls_tests = False
 except ImportError:
     skip_tls_tests = True
+    # In case the tests are skipped because of lacking test harness, get_cert should be
+    # overridden to avoid get_cert failing during test discovery as get_cert is used
+    # in conf_vars decorator
+    get_cert = lambda x: x
 
 DEFAULT_DATE = timezone.datetime(2017, 1, 1)
 SUCCESS_COMMAND = ['airflow', 'tasks', 'run', '--help']
 FAIL_COMMAND = ['airflow', 'tasks', 'run', 'false']
 
+# For now we are temporarily removing Dask support until we get Dask Team help us in making the
+# tests pass again
+skip_dask_tests = True

Review comment:
       It is not. I started discussion on that here https://lists.apache.org/thread/6stgcpjt5jb3xfw92oo1j486j33c8v7m
   
   This is is a second time I start similar discussion - the previous time it was in Jan 2020 https://lists.apache.org/thread/875fpgb7vfpmtxrmt19jmo8d3p6mgqnh and then Dask team chimed in and helped in fixing the tests. 
   
   But more than 1 year later we have similar problem:
   
   I also asked at Dask's disccoure whether they can help again: https://dask.discourse.group/t/potential-removal-of-dask-executor-support-in-airflow/433




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org